forked from mlbuising/seck
-
Notifications
You must be signed in to change notification settings - Fork 0
Dev extractor guide
katiatkn edited this page Nov 3, 2012
·
12 revisions
Extractor extracts various document formats downloaded by the crawler using Apache TIKA library. These includes html, documents, and media types.
Design Plan:
[ExtractorFactory]
|
[Abstract Extractor]
|
--------------------------------------
| | |
[HtmlExtractor] [DocumentExtractor] [MediaExtractor]
Sample code for extracting any of the above document types:
String fileName = "test.docx";
File file = new File(fileName);
ExtractorFactory efactory = new ExtractorFactory();
Extractor ex = efactory.getExtractor(file);
Bloblet b = ex.extract();
Metadata m = b.getMetadata();
HtmlExtractor extracts HyperText Markup Language [text/html]
The class retrieves html title, content, hrefs, and imgs.
DocumentExtractor extracts metadatas for the following document types:
- Microsoft Office document formats [application/msword, application/vnd.ms-powerpoint, vnd.visio, application/vnd.ms-outlook]
- OpenDocument Format [application/vnd.oasis.opendocument.*]
- Portable Document Format [application/pdf]
- Rich Text Format [application/rtf]
- Text formats [text/plain]
Extracts metadata out of images(.jpg, .png, .bmp ) and music (.mp3, .wav) using Apache Tika Library. Some of the metadata is image size, camera type,... Just create an object and call the extract function.
MediaExtractor metadataExtractor = new MediaExtractor();
metadataExtractor.extract(filename);