Skip to content

Commit

Permalink
[CARBONDATA-3051] Fix bugs in unclosed streams
Browse files Browse the repository at this point in the history
Failed to run CarbonReaderTests in windows env. This is due to two
problems in carbondata:

Problem 1: While encountering problems, we should close all the streams instead of
only close current one.

Problem 2: Current FileReader is not singleton nor the fileCache
variable is a class variable, so it can only provide cache function if
we pass the object among the methods. After reading version info, the
fileReader cannot be used by other methods, so we must close it.

This closes #2871
  • Loading branch information
xuchuanyin authored and jackylk committed Oct 31, 2018
1 parent 43ffe79 commit 0e39abf
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 1 deletion.
Expand Up @@ -29,6 +29,7 @@
import org.apache.carbondata.core.metadata.schema.table.CarbonTable;
import org.apache.carbondata.core.scan.expression.Expression;
import org.apache.carbondata.core.util.CarbonSessionInfo;
import org.apache.carbondata.core.util.CarbonUtil;
import org.apache.carbondata.core.util.ThreadLocalSessionInfo;
import org.apache.carbondata.hadoop.api.CarbonFileInputFormat;

Expand Down Expand Up @@ -169,7 +170,7 @@ public <T> CarbonReader<T> build()
reader.initialize(split, attempt);
readers.add(reader);
} catch (Exception e) {
reader.close();
CarbonUtil.closeStreams(readers.toArray(new RecordReader[0]));
throw e;
}
}
Expand Down
Expand Up @@ -93,6 +93,7 @@ public static String getVersionDetails(String dataFilePath) throws IOException {
FileReader fileReader = FileFactory.getFileHolder(FileFactory.getFileType(dataFilePath));
ByteBuffer buffer =
fileReader.readByteBuffer(FileFactory.getUpdatedFilePath(dataFilePath), fileSize - 8, 8);
fileReader.finish();
CarbonFooterReaderV3 footerReader = new CarbonFooterReaderV3(dataFilePath, buffer.getLong());
FileFooter3 footer = footerReader.readFooterVersion3();
if (null != footer.getExtra_info()) {
Expand Down
Expand Up @@ -355,6 +355,7 @@ public void testBlocklet() throws IOException {
FileReader fileReader = FileFactory.getFileHolder(FileFactory.getFileType(dataFile.getPath()));
ByteBuffer buffer = fileReader.readByteBuffer(FileFactory.getUpdatedFilePath(
dataFile.getPath()), dataFile.getSize() - 8, 8);
fileReader.finish();
CarbonFooterReaderV3 footerReader =
new CarbonFooterReaderV3(dataFile.getAbsolutePath(), buffer.getLong());
FileFooter3 footer = footerReader.readFooterVersion3();
Expand Down

0 comments on commit 0e39abf

Please sign in to comment.