JENA-1554, JENA-1555: Support bz2 compressed files directly from Java. #427
Conversation
Add Snappy default 32k block decompress only; compressor not available Update javadoc (RDFLanguages, BinRDF) that mentions gz.
pom.xml
Outdated
| @@ -68,6 +68,7 @@ | |||
| <ver.commonslang3>3.4</ver.commonslang3> | |||
| <ver.commonscsv>1.5</ver.commonscsv> | |||
| <ver.commons-codec>1.11</ver.commons-codec> | |||
| <ver.commons-compress>1.16.1</ver.commons-compress> | |||
There was a problem hiding this comment.
1.17 was just released... maybe worth using it instead? Just received Stefan's announcement message about it in the commons mailing list.
| return filename.substring(0, filename.length()-ext.length()); | ||
| } | ||
| return filename; | ||
| } |
There was a problem hiding this comment.
Maybe instead
/** The filename without any compression extension, or the original filename.
* It tests for compression types handled by {@link #openFileEx}.
*/
static public String filenameNoCompression(String filename) {
if ( FilenameUtils.isExtension(filename, extensions) ) {
return FilenameUtils.removeExtension(filename);
}
return filename;
}I believe we have commons-io already in the dependencies list. There's some extra check for null bytes in the extension check... but that's not so important. Just simpler I think.
| @@ -158,15 +177,18 @@ static public OutputStream openOutputFileEx(String filename) throws FileNotFound | |||
| filename = IRILib.decode(filename) ; | |||
| } | |||
| OutputStream out = new FileOutputStream(filename) ; | |||
| if ( filename.endsWith(".gz") ) | |||
| out = new GZIPOutputStream(out) ; | |||
| String ext = FileOps.extension(filename); | |||
There was a problem hiding this comment.
Digressing; but as we have FilenameUtils.getExtension() in the classpath, from commons-io, perhaps this could later be marked as deprecated?
There was a problem hiding this comment.
Good idea - is it OK to separate out a "clean up FileOps/FileUtils" task and let this PR go in now? Got to finish sometime!
JENA-1555, JENA-1554: Update awaitility ; add Apache Commons compress
JENA-1554: Add bz2 compression/decompression