Kryo memoizer #465
Merged
Kryo memoizer #465
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Rather than writing directly to the memoFile location, write to a tempFile that's located in the same directory and then use File.renameTo to move it. Any errors in writing or renaming will prevent caching.
MetadataStore instances should now be kept in the Memoizer itself and not passed down into further ReaderWrappers. On load and save, copying is done from an OME-XML based impl. It was unclear to me if there were other points at which synchronization between the two needed to take place.
'boolean FormatTools.equalReaders(IFormatReader, IFormatReader)' returns whether or not the two readers are considered to be equivalent. The internal reader wrapper stack and state must be equal in order to return true.
We only need to compare uninitialized state, not the initialized state.
By having the Closeable interface implemented, locations where "close()" is not called in a finally can be detected by findbugs, eclipse, etc.
This reverts commit 0c210b3.
The close method is likely not needed, but is more strictly correct. It was added in an attempt to find the cause of a segfalt.
The finalizer of the java.util.zip.Inflater class was frequently though non-deterministically causing the loading of memo files to segfault the JVM. By marking the field transient, it's no longer being serialized, and therefore not deserialized.
If directory is set, all files will be created as absolute paths under it with all parent paths automatically created. If the directory is set but does not exist, memoization is disabled. This is useful for periodically turning off memos for comparing performance.
|
Rebased to solve grey button issue and to make turbojpeg available on the omero branch |
|
Travis build failing due to Kryo not being added to the relevant pom.xml |
| } | ||
|
|
||
| public void saveVersion(Integer vERSION) { | ||
| kryo.writeObject(output, VERSION); |
There was a problem hiding this comment.
Should this be kryo.writeObject(output, vERSION)?
|
Findbugs still unhappy: http://hudson.openmicroscopy.org.uk/view/Bio-Formats/job/BIOFORMATS-per-commit-test/353/ Opening a PR against this one in a few minutes to hopefully sort that out. |
Otherwise, on deserialization the following exception is
thrown:
```
Exception in thread "Thread-14" java.lang.NullPointerException
at java.util.logging.LogManager.access$900(LogManager.java:130)
at java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:979)
at java.util.logging.LogManager.resetLogger(LogManager.java:677)
at java.util.logging.LogManager.reset(LogManager.java:665)
at java.util.logging.LogManager$Cleaner.run(LogManager.java:223)
```
It's not serializable, and does not offer much in terms of performance improvements.
Works around Kryo issue 100.
Merging in develop in order to fix the travis compilation.
|
Merged in develop to fix genshi. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
loci.formats.Memoizeris a newReaderWrapperimplementatino which memoizes, or cache, all calls toIFormatReader.setId(String)by writing out the internalIFormatReaderdelegate using http://kryo.googlecode.com.Typical usage is to add
Memoizeras one of the outermostReaderWrappers:Files are currently never cleaned up, and exceptions are ignored. Instead, bad memo files should likely be actively deleted.