From acc9f4986380e5fe44bd13359a66520ad47d1507 Mon Sep 17 00:00:00 2001 From: Almeida-a Date: Wed, 31 Aug 2022 16:14:29 +0100 Subject: [PATCH 1/4] Hotfix: bump the maven tag to v0.1.0-a0 (#8) * Tweaked pom (changed artifactID) * Main class added * Reverted artifact ID * Removed name * Tweaked pom * Re-added maven.compiler tags * Tried multiple mvn compiler versions * Removed java 8 from CI (for now) * Re-added java 8 to compiler and ci options * Re-added java 8 to compiler and ci options * Re-added java 8 to ci options * Added test/ci branch to ci scope * Restructured code into the plugin set form * Updated dicoogle version pointer * Template Storage plugin * Implemented interface contract methods * Registered storage plugin * Changed context path name Plus, added some comments: * explaining getJettyHandlers * providing a path example to access a servlet * Fixed target version to 3.0.5 * Fixed target version to 3.0.5 * Simple image viewer (by "get" operation) of uncompressed dicom images * Image store encoding with recent formats (#4) * #3 Introduced ability to transcode the pixel-data into newer (compressed) formats * #3 Complemented transcoding process by adding lossy compression attributes as well as transfer syntax Also fixed problem of file creation in MiscUtils * #3 Added format decoding functions * #3 Fixed multiple bugs * Thread waits for compression processes to finish * Files generated by the encoders are now deleted upon exit * Added logic for detecting and avoiding duplicate file storing (although, this may already be enforced by dicoogle itself) * #3 decode bugfix: thread waits for decompression subprocess to finish * Viewer for encoded dicom images. (#6) * #5 decode feature: provided method to retrieve buffered image from a dicom object with an encoded pixel-data * #5 image viewer: finalized the feature Also: other minor fixes and tweaks. * #5 image viewer: Added configuration to choose the format Also: fixed the "davif" command decode - using avif_decode (rust) instead * Bump maven version tag to v0.1.0-a0 --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 5926066..514857f 100644 --- a/pom.xml +++ b/pom.xml @@ -6,7 +6,7 @@ pt.ua.ieeta imodec-dicoogle-plugin-set imodec-dicoogle-plugin-set - 0.0.0 + 0.1.0-a0 jar From fb83bf23fed609c755b2136e72c304cdfbd7cd6a Mon Sep 17 00:00:00 2001 From: Almeida-a Date: Wed, 21 Sep 2022 12:49:02 +0100 Subject: [PATCH 2/4] Bump to 0.1.0-a1 (#23) * Allow storing pixel-data in native format (#17) * #14 Feature: Allow for native transfer syntaxes By default or if specified * #14 Refactor: organized the enums into a single package & cherry-picked the readme from main & changed the 'same' keyword that is to keep the transfer syntax to 'keep' * #14 Feature: Enable saving all transfer syntaxes at once By all TS, it is meant all new formats' TSs, as well as the native version (as long the original format is not lossy) * #14 Feature: Enable viewing any available TS version of a dicom object * Using the tsuid or codec parameter - if left unspecified, then use native form * TS - Transfer Syntax(es) * #14 Docs: Updates on the viewer http request parameters * Fixed readme merge mistake * Docs: Add table of contents to the readme * #14 Fix: Adding 'all' to the codec config tag * Create CODE_OF_CONDUCT.md (#18) * Update issue templates * Added contributing section to readme * Allow defining encoding parameters (#19) * #15 Feature: quality and speed parameters available However, still beyond reach of configuration by the user Also: error handling for subprocesses * #15 fix: merge mistake * #15 refactor: inlined encode function into encodePNGFile Also, encoding options are now set to default * #15 Feature: Default encoding options are now extracted from yml From encoding-options.yaml Also, quality value number type of each encoder is now specified in NewFormat enum Also, removed unused code * #15 Feature: Allow defining quality and speed encoding parameters Configuration is performed at DicoogleDir/Plugins/settings/imodec-plugin-set.xml * #15 Feature: Warning when encoding parameters are invalid * Support for storing multi-frame images* (#22) * #11 Refactor: Paved way to start developing support for multi-frame dicom objects * #11 Refactor: Memory optimization when codec -> all Optimization is to use iterator, for each dicom object clone, instead of array - more memory efficient Also, fixed bug when getting uri (was not getting the correct TS sometimes) Also, changed some dependencies * #11 Feature: Store multi-frame images * #11 Finished feature*: Viewing raw multi-frame images as gif * Workaround - Could not find a way yet to display animated images. Thus, server can only view them locally by a file:// link * I marked the source of the above problem with a 'fixme 16-09-22' tag * #11 Finished feature: Multi-frame store operation * #11 [Bugfix] Some old features that were broken By the multi-frame support feature * Code cleanup * [Refactor] Moves functions pertaining utility to *Utils.java * #11 [Bugfix] could not read single frame avif * #11 [README] Informed that `all` does not work with multi-frames storing operation --- .github/ISSUE_TEMPLATE/bug_report.md | 38 +++ .github/ISSUE_TEMPLATE/feature_request.md | 20 ++ CODE_OF_CONDUCT.md | 128 ++++++++ README.md | 66 +++- pom.xml | 60 +++- .../java/pt/ua/imodec/ImodecPluginSet.java | 108 ++++++- .../imodec/storage/ImodecStoragePlugin.java | 91 ++++-- .../java/pt/ua/imodec/util/DicomUtils.java | 301 +++++++++++++++++- .../java/pt/ua/imodec/util/FrameIterator.java | 39 +++ .../pt/ua/imodec/util/GifSequenceWriter.java | 193 +++++++++++ .../java/pt/ua/imodec/util/ImageUtils.java | 128 +++----- .../java/pt/ua/imodec/util/MiscUtils.java | 184 ++++++++++- .../java/pt/ua/imodec/util/NewFormat.java | 50 --- .../pt/ua/imodec/util/NewFormatsCodecs.java | 121 +++---- .../pt/ua/imodec/util/formats/Format.java | 10 + .../pt/ua/imodec/util/formats/Native.java | 29 ++ .../pt/ua/imodec/util/formats/NewFormat.java | 95 ++++++ .../validators/EncodeabilityValidator.java | 29 ++ .../imodec/util/validators/OSValidator.java | 2 +- .../ua/imodec/util/validators/Validator.java | 9 + .../imodec/webservice/ImodecJettyPlugin.java | 18 +- .../webservice/ImodecJettyWebService.java | 164 ++++++---- src/main/resources/encoding-options.yaml | 9 + src/main/resources/viewer-template.html | 10 + 24 files changed, 1604 insertions(+), 298 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/bug_report.md create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md create mode 100644 CODE_OF_CONDUCT.md create mode 100644 src/main/java/pt/ua/imodec/util/FrameIterator.java create mode 100644 src/main/java/pt/ua/imodec/util/GifSequenceWriter.java delete mode 100644 src/main/java/pt/ua/imodec/util/NewFormat.java create mode 100644 src/main/java/pt/ua/imodec/util/formats/Format.java create mode 100644 src/main/java/pt/ua/imodec/util/formats/Native.java create mode 100644 src/main/java/pt/ua/imodec/util/formats/NewFormat.java create mode 100644 src/main/java/pt/ua/imodec/util/validators/EncodeabilityValidator.java create mode 100644 src/main/java/pt/ua/imodec/util/validators/Validator.java create mode 100644 src/main/resources/encoding-options.yaml create mode 100644 src/main/resources/viewer-template.html diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..dd84ea7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,38 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '' +labels: '' +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Desktop (please complete the following information):** + - OS: [e.g. iOS] + - Browser [e.g. chrome, safari] + - Version [e.g. 22] + +**Smartphone (please complete the following information):** + - Device: [e.g. iPhone6] + - OS: [e.g. iOS8.1] + - Browser [e.g. stock browser, safari] + - Version [e.g. 22] + +**Additional context** +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..bbcbbe7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,20 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '' +labels: '' +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..543d548 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,128 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, religion, or sexual identity +and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +* Focusing on what is best not just for us as individuals, but for the + overall community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or + advances of any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email + address, without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, and will communicate reasons for moderation +decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of representing our community include using an official e-mail address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the community leaders responsible for enforcement at +e-mail. +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series +of actions. + +**Consequence**: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction with +those enforcing the Code of Conduct, for a specified period of time. This +includes avoiding interactions in community spaces as well as external channels +like social media. Violating these terms may lead to a temporary or +permanent ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including +sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within +the community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 2.0, available at +https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. + +Community Impact Guidelines were inspired by [Mozilla's code of conduct +enforcement ladder](https://github.com/mozilla/diversity). + +[homepage]: https://www.contributor-covenant.org + +For answers to common questions about this code of conduct, see the FAQ at +https://www.contributor-covenant.org/faq. Translations are available at +https://www.contributor-covenant.org/translations. diff --git a/README.md b/README.md index 594da80..d8df810 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,20 @@ Imodec (Image [Modern] Codecs) is a set of plugins for the [Dicoogle](https://github.com/bioinformatics-ua/dicoogle/) project providing the services of modern image compression codecs. +## Table of contents +1. [Building from source](#building-from-source) +2. [How to use](#how-to-use) + 1. [Pre-requisites](#pre-requisites) + 2. [Plugging into dicoogle](#plugging-into-dicoogle) + 3. [Store-SCU operation](#store-scu-operation) + 1. [storescu - dcmtk](#storescu---dcmtk) + 2. [dicom-storescu - dicom-rs](#dicom-storescu---dicom-rs) + 4. [View the resulting images](#view-the-resulting-images) + 1. [Http Request Structure](#http-request-structure) +3. [Other Notes](#other-notes) + 1. [New transfer syntaxes](#new-transfer-syntaxes) + 2. [Contributing](#contributing) + 3. [Configuring encoding options](#configuring-encoding-options) ## Building from source If you want, you can build from source using the `mvn` @@ -57,7 +71,9 @@ xml settings file (path `Plugins/settings/imodec-plugin-set.xml`): jxl ``` -Possible values are: `jxl`, `avif` and `webp`. +Possible values are: `jxl`, `avif`, `webp`, `keep` and `all` for all the previous options simultaneously. + +Note: Multi-frame images are not expected to work in the `all` setting. You need to use a specific tool for the store operation. @@ -92,6 +108,52 @@ In order to check the stored images, you need to input an url to your browser with the SOP Instance UID of the respective dicom object, following the next example: +```http request +http://localhost:8080/imodec/view?siuid=2.25.69906150082773205181031737615574603347&codec=jxl ``` -http://localhost:8080/imodec/view?siuid=2.25.69906150082773205181031737615574603347 + +#### Http request structure +Base url: +```http request +http://localhost:8080/imodec/view ``` + +Parameters: + * `siuid` [Required]: SOP Instance UID of the dicom object's image to be viewed. + * `tsuid` [Optional]: Transfer Syntax UID defining a version of the dicom object in a specific format. + * `codec` [Optional]: If you want to see the image of a specific modern format, choose here which format that +you want to see. This is the same as choosing the [transfer syntax](#new-transfer-syntaxes) of that specific codec with the +above parameter. If both are used, `tsuid` overrides `codec`. + + +## Other Notes + +### New Transfer Syntaxes + +The new image formats will encode the pixel data of the dicom objects. +The transfer syntaxes define the format of the pixel-data of the dicom objects. +Thus, new transfer syntaxes are created to define pixel-data with the bitstream of those modern codecs. + +New Transfer-Syntax list: + * JPEG-XL: `1.2.826.0.1.3680043.2.682.104.1` + * WebP: `1.2.826.0.1.3680043.2.682.104.2` + * AVIF: `1.2.826.0.1.3680043.2.682.104.3` + +### Contributing +This project encompasses developing a set of plugins for the dicoogle software. Therefore, for anyone interested in contributing, imodec follows the [dicoogle development guidelines](https://github.com/bioinformatics-ua/dicoogle/wiki#development-guidelines). + +### Configuring encoding options + +This is a more advanced configuration. +You can define encoding options such as quality or speed of compression +(depending on the name of the configuration parameters). + +An example of those options is displayed below: +```xml + + ... + + + + +``` \ No newline at end of file diff --git a/pom.xml b/pom.xml index 514857f..22bc708 100644 --- a/pom.xml +++ b/pom.xml @@ -12,13 +12,12 @@ UTF-8 - + 3.1.0 - - 9.4.48.v20220622 - - 5.23.3 + 5.27.0 + 1.32 + 1.1 @@ -34,12 +33,57 @@ jetty-rewrite ${jetty.version} - + + + org.yaml + snakeyaml + ${snakeyaml.version} + + + jaidcm4che + jai_imageio + 1.1 + + + jai_core + javax.media + + + + + commons-io + commons-io + 2.11.0 + + + + com.twelvemonkeys.imageio + imageio-core + 3.8.3 + + + + com.twelvemonkeys.imageio + imageio-jpeg + 3.8.3 + + + com.twelvemonkeys.servlet + servlet + 3.8.3 + + org.dcm4che - dcm4che-core - 5.27.0 + dcm4che-imageio + ${dcm4che.version} + + + + + + diff --git a/src/main/java/pt/ua/imodec/ImodecPluginSet.java b/src/main/java/pt/ua/imodec/ImodecPluginSet.java index b5fc663..77dd8b5 100644 --- a/src/main/java/pt/ua/imodec/ImodecPluginSet.java +++ b/src/main/java/pt/ua/imodec/ImodecPluginSet.java @@ -1,6 +1,8 @@ package pt.ua.imodec; import net.xeoh.plugins.base.annotations.PluginImplementation; +import org.apache.commons.configuration.tree.ConfigurationNode; +import org.dcm4che2.data.TransferSyntax; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import pt.ua.dicoogle.sdk.JettyPluginInterface; @@ -8,12 +10,14 @@ import pt.ua.dicoogle.sdk.StorageInterface; import pt.ua.dicoogle.sdk.settings.ConfigurationHolder; import pt.ua.imodec.storage.ImodecStoragePlugin; -import pt.ua.imodec.util.NewFormat; +import pt.ua.imodec.util.formats.Format; +import pt.ua.imodec.util.formats.Native; +import pt.ua.imodec.util.formats.NewFormat; import pt.ua.imodec.webservice.ImodecJettyPlugin; -import java.util.Arrays; -import java.util.Collection; -import java.util.Collections; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.*; /** * @@ -38,16 +42,25 @@ public class ImodecPluginSet implements PluginSet { private final ImodecStoragePlugin storage; // Additional resources - public static NewFormat chosenFormat = null; + public static Format chosenFormat = null; + public static final Path TMP_DIR_PATH = Paths.get("/tmp/imodec"); + private ConfigurationHolder settings; public ImodecPluginSet() { this.jettyWeb = new ImodecJettyPlugin(); this.storage = new ImodecStoragePlugin(); + tmpMkdirs(); + logger.info("Imodec Plugin Set is ready"); } + private static void tmpMkdirs() { + if (!TMP_DIR_PATH.toFile().mkdirs()) + logger.info("Could not create main tmp directory. It already exists"); + } + @Override public String getName() { return "imodec-plugin-set"; @@ -66,25 +79,94 @@ public Collection getStoragePlugins() { @Override public void setSettings(ConfigurationHolder xmlSettings) { - if (chosenFormat == null) + if (chosenFormat == null) { setImageCompressionFormat(xmlSettings); + setEncoderOptions(xmlSettings); + } this.settings = xmlSettings; } + private void setEncoderOptions(ConfigurationHolder xmlSettings) { + + List configurationNodeList = xmlSettings + .getConfiguration() + .getRootNode().getChildren(); + + for (ConfigurationNode tag : + configurationNodeList) { + Optional format = Arrays.stream(NewFormat.values()) + .filter(format1 -> format1.getFileExtension().equals(tag.getName())) + .findFirst(); + + if (!format.isPresent()) + continue; + + NewFormat format1 = format.get(); + Optional attributeQuality = tag.getAttributes() + .stream() + .filter(configurationNode -> configurationNode.getName().equals(format1.getQualityParamName())) + .findFirst(); + Optional attributeSpeed = tag.getAttributes() + .stream() + .filter(configurationNode -> configurationNode.getName().equals(format1.getSpeedParamName())) + .findFirst(); + + if (attributeQuality.isPresent()) { + try { + Float value = Float.valueOf((String) attributeQuality.get().getValue()); + logger.debug("Format '{}' quality option '{}' was changed to '{}'", + format1.getId(), format1.getQualityParamValue(), value); + format1.setQualityParamValue(value); + } catch (ClassCastException | NumberFormatException ignored) { + logger.warn("Invalid quality value for '{}' -> '{}'." + + " Maintaining default options.", + format1.getFileExtension(), attributeQuality.get().getValue()); + } + } + + if (attributeSpeed.isPresent()) { + try { + Float value = Float.valueOf((String) attributeSpeed.get().getValue()); + logger.debug("Format '{}' speed option '{}' was changed to '{}'", + format1.getId(), format1.getSpeedParamValue(), value); + format1.setSpeedParamValue(value); + } catch (ClassCastException | NumberFormatException ignored) { + logger.warn("Invalid speed value for '{}' -> '{}'." + + " Maintaining default options.", + format1.getFileExtension(), attributeSpeed.get().getValue()); + } + } + } + } + private static void setImageCompressionFormat(ConfigurationHolder xmlSettings) { - NewFormat defaultFormat = NewFormat.JPEG_XL; - String chosenFormatExtension = xmlSettings.getConfiguration().getString("codec"); + Format defaultFormat = Native.UNCHANGED; + + String chosenFormatId = xmlSettings.getConfiguration().getString("codec"); + + if (chosenFormatId.equals("all")) { + chosenFormat = new Format() { + @Override + public TransferSyntax getTransferSyntax() { + return null; + } - chosenFormat = Arrays.stream(NewFormat.values()) - .filter(newFormat -> newFormat.getFileExtension().equals(chosenFormatExtension)) - .findFirst() - .orElse(defaultFormat); + @Override + public String getId() { + return "all"; + } + }; + } else + chosenFormat = Arrays.stream(((Format[]) NewFormat.values())) + .filter(newFormat -> newFormat.getId().equals(chosenFormatId)) + .findFirst() + .orElse(defaultFormat); logger.info( String.format("Format requested: '%s', set -> '%s'", - chosenFormatExtension, chosenFormat.getFileExtension()) + chosenFormatId, chosenFormat.getId()) ); } diff --git a/src/main/java/pt/ua/imodec/storage/ImodecStoragePlugin.java b/src/main/java/pt/ua/imodec/storage/ImodecStoragePlugin.java index 8d4fee1..35ca34a 100644 --- a/src/main/java/pt/ua/imodec/storage/ImodecStoragePlugin.java +++ b/src/main/java/pt/ua/imodec/storage/ImodecStoragePlugin.java @@ -10,35 +10,39 @@ import pt.ua.dicoogle.sdk.StorageInterface; import pt.ua.dicoogle.sdk.settings.ConfigurationHolder; import pt.ua.imodec.ImodecPluginSet; -import pt.ua.imodec.util.ImageUtils; +import pt.ua.imodec.util.DicomUtils; import pt.ua.imodec.util.MiscUtils; -import pt.ua.imodec.util.NewFormat; +import pt.ua.imodec.util.formats.Format; +import pt.ua.imodec.util.formats.Native; +import pt.ua.imodec.util.formats.NewFormat; import java.io.*; import java.net.URI; -import java.util.ArrayList; -import java.util.Collection; -import java.util.HashMap; -import java.util.NoSuchElementException; +import java.util.*; import java.util.function.Supplier; /** * - * Basic Storage Plugin + * Storage Plugin * - "Template" from rlebre/dicoogle-plugin-sample *

* */ public class ImodecStoragePlugin implements StorageInterface { private static final Logger logger = LoggerFactory.getLogger(ImodecStoragePlugin.class); + private static final String scheme = "imodec-mem"; - private final HashMap mem = new HashMap<>(); + private final HashMap mem = new HashMap<>(); // TODO: 16/09/22 Optimize: Refactor BAOS to OS, to allow for using other output streams - very good for memory optimization private boolean enabled = true; private ConfigurationHolder settings; @Override public String getScheme() { - return "imodec-mem"; + return scheme; + } + + public boolean containsURI(final URI uri) { + return mem.containsKey(uri.toString()); } @Override @@ -53,7 +57,7 @@ public URI getURI() { } @Override - public InputStream getInputStream() { + public InputStream getInputStream() throws IOException { ByteArrayOutputStream bos = mem.get(location.toString()); if (bos == null) @@ -61,7 +65,13 @@ public InputStream getInputStream() { String.format("File uri='%s' was not found at the storage!", location) ); - return new ByteArrayInputStream(bos.toByteArray()); + try { + return new ByteArrayInputStream(bos.toByteArray()); + } catch (OutOfMemoryError ignored) { + logger.info("Large bitstream object encountered. " + + "Changing approach for data retrieval..."); + return MiscUtils.getInputStreamFromLarge(bos); + } } @Override @@ -77,20 +87,42 @@ public long getSize() { @Override public URI store(DicomObject dicomObject, Object... objects) { - logger.warn("Waiting while format is being set"); - - Supplier choosingProcess = () -> ImodecPluginSet.chosenFormat == null; - MiscUtils.sleepWhile(choosingProcess); - NewFormat chosenFormat = ImodecPluginSet.chosenFormat; - - URI uri = URI.create(getScheme() + "://" + dicomObject.getString(Tag.SOPInstanceUID)); + URI uri = getUri(dicomObject); if (mem.containsKey(uri.toString())) { logger.warn("This object was already stored!"); return uri; } + logger.info("Waiting while format is being set"); + Supplier choosingProcess = () -> ImodecPluginSet.chosenFormat == null; + MiscUtils.sleepWhile(choosingProcess); + Format chosenFormat = ImodecPluginSet.chosenFormat; + boolean encodeWithAllTS = chosenFormat.getId().equals("all"); + boolean zerothLevelRecursion = objects.length == 0; + + try { - ImageUtils.encodeDicomObject(dicomObject, chosenFormat); + if (chosenFormat instanceof NewFormat) { + + DicomUtils.encodeDicomObject(dicomObject, (NewFormat) chosenFormat, new HashMap<>()); + + } else if (DicomUtils.isMultiFrame(dicomObject) + && encodeWithAllTS && zerothLevelRecursion) { + + logger.warn("This is not memory optimized. Memory errors are prone to occur."); + File dicomObjectFile = DicomUtils.writeDicomObjectToTmpFile(dicomObject); + Iterator dicomInputStreamIterator = DicomUtils.encodeIteratorDicomInputStreamWithAllTs(dicomObjectFile); + while (dicomInputStreamIterator.hasNext()) { + store(dicomInputStreamIterator.next(), Native.UNCHANGED); + } + + } else if (encodeWithAllTS && zerothLevelRecursion) { // Same as previous but single frame + // TODO: 03/09/22 Find a better way for stopping condition than by the number of argument objects + Iterator dicomObjectsIterator = DicomUtils.encodeIteratorDicomObjectWithAllTs(dicomObject); + while (dicomObjectsIterator.hasNext()) { + store(dicomObjectsIterator.next(), Native.UNCHANGED); + } + } } catch (IOException e) { throw new RuntimeException(e); } @@ -102,15 +134,30 @@ public URI store(DicomObject dicomObject, Object... objects) { } catch (IOException ex) { logger.warn("Failed to store object", ex); } - bos.toByteArray(); mem.put(uri.toString(), bos); + logger.info("Object successfully stored!"); return uri; } + private URI getUri(DicomObject dicomObject) { + + String tsUID; + Format chosenFormat = ImodecPluginSet.chosenFormat; + + if (chosenFormat.equals(Native.UNCHANGED) || chosenFormat.getId().equals("all")) + tsUID = dicomObject.getString(Tag.TransferSyntaxUID); + else + tsUID = chosenFormat.getTransferSyntax().uid(); + + return URI.create(getScheme() + "://" + + dicomObject.getString(Tag.SOPInstanceUID) + "/" + + tsUID); + } + @Override public URI store(DicomInputStream dicomInputStream, Object... objects) throws IOException { - return store(dicomInputStream.readDicomObject()); + return store(dicomInputStream.readDicomObject(), objects); } @Override @@ -120,7 +167,7 @@ public void remove(URI location) { @Override public String getName() { - return "imodec-storage"; + return "imodec-storage-plugin"; } @Override diff --git a/src/main/java/pt/ua/imodec/util/DicomUtils.java b/src/main/java/pt/ua/imodec/util/DicomUtils.java index 34119f6..a6e810e 100644 --- a/src/main/java/pt/ua/imodec/util/DicomUtils.java +++ b/src/main/java/pt/ua/imodec/util/DicomUtils.java @@ -1,12 +1,28 @@ package pt.ua.imodec.util; -import org.dcm4che2.data.DicomObject; +import org.apache.commons.lang.SerializationUtils; +import org.dcm4che2.data.*; +import org.dcm4che2.io.DicomInputHandler; +import org.dcm4che2.io.DicomInputStream; import org.dcm4che2.io.DicomOutputStream; +import org.dcm4che2.io.StopTagInputHandler; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import pt.ua.imodec.ImodecPluginSet; +import pt.ua.imodec.util.formats.NewFormat; +import pt.ua.imodec.util.validators.EncodeabilityValidator; +import pt.ua.imodec.util.validators.OSValidator; +import javax.imageio.ImageIO; +import javax.imageio.ImageReadParam; +import javax.imageio.ImageReader; +import javax.imageio.stream.ImageInputStream; +import java.awt.image.BufferedImage; import java.io.File; import java.io.IOException; +import java.nio.file.Files; +import java.rmi.UnexpectedException; +import java.util.*; public class DicomUtils { @@ -16,13 +32,18 @@ public class DicomUtils { * Saves dicom object to a file * * @param dicomObject dicom data to save - * @param dicomFile Path of the file that will be created - * @return Whether if the operation is successful or not */ - public static boolean saveDicomFile(DicomObject dicomObject, File dicomFile, boolean temporary) throws IOException { + public static File saveDicomFile(DicomObject dicomObject, boolean temporary) throws IOException { - if (dicomFile.exists()) - logger.warn("File '" + dicomFile.getAbsolutePath() + "' already exists! Overwriting"); + String tmpDicomFileName = String.format("%s/DicomUtils/%s.dcm", ImodecPluginSet.TMP_DIR_PATH, + dicomObject.getString(Tag.SOPInstanceUID)); + File dicomFile = new File(tmpDicomFileName); + dicomFile.deleteOnExit(); + + if (dicomFile.exists()) { + logger.debug("File '" + dicomFile.getAbsolutePath() + "' already exists! No operation."); + return dicomFile; + } if (!MiscUtils.createNewFile(dicomFile, true)) throw new IllegalStateException( @@ -34,9 +55,275 @@ public static boolean saveDicomFile(DicomObject dicomObject, File dicomFile, boo try (DicomOutputStream outputStream = new DicomOutputStream(dicomFile)) { outputStream.writeDicomFile(dicomObject); - return true; } + return dicomFile; + + } + + public static DicomObject readNonPixelData(DicomInputStream dicomInputStream) throws IOException { + DicomInputHandler nonPixelDataHandler = new StopTagInputHandler(Tag.PixelData); + + dicomInputStream.setHandler(nonPixelDataHandler); + + return dicomInputStream.readDicomObject(); + } + + public static boolean isMultiFrame(DicomObject dicomObject) { + return dicomObject.contains(Tag.NumberOfFrames) && dicomObject.getInt(Tag.NumberOfFrames) > 1; + } + + static BufferedImage loadDicomEncodedFrame(DicomInputStream inputStream, int frameID, NewFormat newFormat) throws IOException { + // FIXME: 17/09/22 This readDicomObject is an OOM hazard. + // Use DicomInputStream to read the frames w/o loading them all to memory + + DicomObject dicomObject = inputStream.readDicomObject(); + DicomElement frameSequence = dicomObject.get(Tag.PixelData); + + if (frameSequence.vr().equals(VR.SQ)) + throw new AssertionError("Tried to load a frame from a non multi-frame dicom object!"); + + byte[] codeStream = frameSequence.getFragment(frameID); + + Optional image = Optional.ofNullable( + NewFormatsCodecs.decodeByteStream(codeStream, newFormat)); + if (!image.isPresent()) + throw new NullPointerException("Error reading frame!"); + return image.get(); + } + + /** + * Load dicom (buffered) image. + * Credits: + * Source. + * + * @param inputStream Stream with the dicom data + * @param frame ordinal value of the frame to be retrieved, if image is single frame, then always 0 + * @return The buffered image + * @throws IOException if the image format is not DICOM or another IO issue occurred + */ + public static BufferedImage loadDicomImage(DicomInputStream inputStream, int frame) throws IOException { + try (ImageInputStream imageInputStream = ImageIO.createImageInputStream(inputStream)) { + ImageReader reader = ImageUtils.getImageReader("DICOM"); + ImageReadParam param = reader.getDefaultReadParam(); + reader.setInput(imageInputStream, false); + BufferedImage image = reader.read(frame, param); + if (image == null) + throw new NullPointerException("Error reading dicom image!"); + return image; + } } + public static void encodeDicomObject( + DicomObject dicomObject, NewFormat chosenFormat, + HashMap options) throws IOException { + + if (isMultiFrame(dicomObject)) + encodeMultiFrameDicomObject(dicomObject, chosenFormat, options); + else + encodeSingleFrameDicomObject(dicomObject, chosenFormat, options); + + } + + static void encodeMultiFrameDicomObject(DicomObject dicomObject, NewFormat chosenFormat, + HashMap options) throws IOException { + int framesSqLength = 0; + + ImageUtils.logger.info("Encoding multi-frame dicom object with '{}' format. This might take a while...", + chosenFormat.getFileExtension()); + + if (!EncodeabilityValidator.validate(dicomObject)) { + ImageUtils.logger.error("Cannot encode this dicom object!"); + throw new UnsupportedOperationException(); + } + + if (!OSValidator.validate()) { + throw new IllegalStateException( + String.format("Unsupported OS: '%s' for Imodec storage plugin", System.getProperty("os.name")) + ); + } + + Iterator frameIterator = new FrameIterator(dicomObject); + + ImageUtils.logger.debug("Creating directory to store the (multi-frame) image frames"); + String sopInstUID = dicomObject.getString(Tag.SOPInstanceUID); + File imageDir = new File(String.format("%s/%s", ImodecPluginSet.TMP_DIR_PATH, sopInstUID)); + if (!imageDir.exists() && !imageDir.mkdirs()) { + throw new UnexpectedException("Could not create directory!"); + } + imageDir.deleteOnExit(); + + int nativePixelDataSizeInBytes; + DicomElement pixelDataElement = dicomObject.get(Tag.PixelData); + if (pixelDataElement.vr().equals(VR.SQ)) + nativePixelDataSizeInBytes = pixelDataElement.getBytes().length; + else + nativePixelDataSizeInBytes = pixelDataElement.length(); + + int frameCounter = 0; + + while (frameIterator.hasNext()) { + BufferedImage frame = frameIterator.next(); + + ImageUtils.logger.debug("Storing image frame into png file."); + File framePNG = new File(String.format("%s/%d.png", imageDir, frameCounter++)); + framePNG.deleteOnExit(); + ImageIO.write(frame, "png", framePNG); + + ImageUtils.logger.debug("Adding frame bitstream into pixel data"); + byte[] codeStream = NewFormatsCodecs.encodePNGFile(framePNG, chosenFormat, options); + framesSqLength += codeStream.length; + + } + + TransferSyntax dicomObjectsTS = TransferSyntax.valueOf(dicomObject.getString(Tag.TransferSyntaxUID)); + + ImageUtils.logger.debug("Emptying pixel-data (re-writing with the sum of the length of all frames or -1)."); + DicomElement dicomFramesSequence; + if (dicomObjectsTS.explicitVR()) + dicomFramesSequence = dicomObject.putSequence(Tag.PixelData, framesSqLength); + else + dicomFramesSequence = dicomObject.putSequence(Tag.PixelData); + + ImageUtils.logger.debug("Inserting the encoded frames onto the pixel data sequence"); + BasicDicomObject sequenceDicomObject = new BasicDicomObject(); + for (File encodedFrame : Objects.requireNonNull(imageDir.listFiles(file -> file.getName().endsWith(".png")))) { + byte[] bitstream = Files.readAllBytes(encodedFrame.toPath()); + sequenceDicomObject.putBytes(Tag.Item, VR.OB, bitstream); + } + if (!dicomObjectsTS.explicitVR()) + sequenceDicomObject.putNull(Tag.SequenceDelimitationItem, VR.OB); + dicomFramesSequence.addDicomObject(sequenceDicomObject); + + ImageUtils.logger.debug("Changing parameters other than pixel-data (lossy compression, transfer syntax, ...)"); + int compressedPixelDataSequenceSizeInBytes = framesSqLength; + updateLossyAttributes(dicomObject, chosenFormat, nativePixelDataSizeInBytes, + compressedPixelDataSequenceSizeInBytes); + } + + private static void updateLossyAttributes(DicomObject dicomObject, NewFormat chosenFormat, + int nativePixelDataSizeInBytes, int compressedPixelDataSizeInBytes) { + dicomObject.putString(Tag.TransferSyntaxUID, VR.UI, chosenFormat.getTransferSyntax().uid()); + dicomObject.putString(Tag.LossyImageCompression, VR.CS, "01"); + dicomObject.putString(Tag.LossyImageCompressionRatio, VR.DS, + String.valueOf(nativePixelDataSizeInBytes / compressedPixelDataSizeInBytes)); + dicomObject.putString(Tag.LossyImageCompressionMethod, VR.CS, chosenFormat.getMethod()); + } + + private static void encodeSingleFrameDicomObject(DicomObject dicomObject, NewFormat chosenFormat, HashMap options) throws IOException { + ImageUtils.logger.info("Encoding single-frame dicom object with '{}' format...", + chosenFormat.getFileExtension()); + + if (!EncodeabilityValidator.validate(dicomObject)) { + ImageUtils.logger.error("Cannot encode this dicom object!"); + throw new UnsupportedOperationException(); + } + + if (!OSValidator.validate()) + throw new IllegalStateException( + String.format("Unsupported OS: '%s' for Imodec storage plugin", System.getProperty("os.name")) + ); + + int rawImageByteSize = dicomObject.getBytes(Tag.PixelData).length; + + // Change dicom object from uncompressed to JPEG XL, WebP or AVIF format + BufferedImage dicomImage = ImageUtils.loadDicomImage(dicomObject, 0); + + ImageUtils.logger.debug("Storing image into png file."); + File tmpImageFile = new File(String.format("%s/%d.png", ImodecPluginSet.TMP_DIR_PATH, dicomImage.hashCode())); + tmpImageFile.deleteOnExit(); + ImageIO.write(dicomImage, "png", tmpImageFile); + + byte[] bitstream = NewFormatsCodecs.encodePNGFile(tmpImageFile, chosenFormat, options); + int compressedImageByteSize = bitstream.length; + + // Adding the new data into the dicom object + dicomObject.putBytes(Tag.PixelData, VR.OB, bitstream); + // Change parameters other than pixel-data (lossy compression, transfer syntax, ...) + updateLossyAttributes(dicomObject, chosenFormat, rawImageByteSize, compressedImageByteSize); + } + + public static Iterator encodeIteratorDicomObjectWithAllTs(DicomObject dicomObject) { + if (dicomObject.contains(Tag.LossyImageCompression) + && dicomObject.getString(Tag.LossyImageCompression).equals("01")) + throw new IllegalArgumentException("Cannot re-apply lossy compression to image!"); + + Iterator newFormats = Arrays.stream(NewFormat.values()).iterator(); + + return new Iterator() { + @Override + public boolean hasNext() { + return newFormats.hasNext(); + } + + @Override + public DicomObject next() { + try { + DicomObject dicomObjectClone = (DicomObject) SerializationUtils.clone(dicomObject); + encodeDicomObject(dicomObjectClone, newFormats.next(), new HashMap<>()); + return dicomObjectClone; + } catch (IOException e) { + ImageUtils.logger.error("Unexpected error!"); + throw new RuntimeException(e); + } catch (OutOfMemoryError error) { + ImageUtils.logger.error("File is too big!"); + throw new RuntimeException(error); + } + } + }; + } + + /** + * + * @param dicomObjectFile Assumed to be multi-frame + * @return Iterator with input stream objects + */ + public static Iterator encodeIteratorDicomInputStreamWithAllTs(File dicomObjectFile) { + + Iterator newFormatIterator = Arrays.stream(NewFormat.values()).iterator(); + + return new Iterator() { + @Override + public boolean hasNext() { + return newFormatIterator.hasNext(); + } + + @Override + public DicomInputStream next() { + try { + ImageUtils.logger.debug("Fetching dicom object from file"); + DicomInputStream dicomInputStream = new DicomInputStream(dicomObjectFile); + DicomObject dicomObject = dicomInputStream.readDicomObject(); // FIXME: 20/09/22 OOM Hazard + dicomInputStream.close(); + + encodeMultiFrameDicomObject(dicomObject, newFormatIterator.next(), new HashMap<>()); + File file1 = writeDicomObjectToTmpFile(dicomObject); + return new DicomInputStream(file1); + } catch (IOException e) { + throw new RuntimeException(e); + } catch (OutOfMemoryError error) { + ImageUtils.logger.error("Multi-frame dicom object is too big!"); + throw new RuntimeException(error); + } + } + }; + } + + public static File writeDicomObjectToTmpFile(DicomObject dicomObject) throws IOException { + String tmpFileName = String.format("%s/tmp%s-%s.dcm", ImodecPluginSet.TMP_DIR_PATH, + dicomObject.getString(Tag.SOPInstanceUID), dicomObject.getString(Tag.TransferSyntaxUID)); + + File file = new File(tmpFileName); + if (!file.exists()) { + boolean ignored = file.createNewFile(); + + ImageUtils.logger.debug("Writing dicom object to file"); + DicomOutputStream dicomOutputStream = new DicomOutputStream(file); + dicomOutputStream.writeDicomFile(dicomObject); + dicomOutputStream.close(); + } + file.deleteOnExit(); + return file; + } } diff --git a/src/main/java/pt/ua/imodec/util/FrameIterator.java b/src/main/java/pt/ua/imodec/util/FrameIterator.java new file mode 100644 index 0000000..ee670c5 --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/FrameIterator.java @@ -0,0 +1,39 @@ +package pt.ua.imodec.util; + +import org.dcm4che2.data.DicomObject; +import org.dcm4che2.data.Tag; + +import java.awt.image.BufferedImage; +import java.io.IOException; +import java.util.Iterator; + +public class FrameIterator implements Iterator { + + private int frame_i = 0; + private final DicomObject dicomObject; + + public FrameIterator(DicomObject dicomObject) { + this.dicomObject = dicomObject; + } + + @Override + public boolean hasNext() { + return frame_i + 1 < dicomObject.getInt(Tag.NumberOfFrames); + } + + @Override + public BufferedImage next() { + BufferedImage frame; + + // Retrieve the frame + try { + frame = ImageUtils.loadDicomImage(dicomObject, frame_i); + } catch (IOException e) { + throw new RuntimeException(e); + } + + frame_i++; + return frame; + } + +} diff --git a/src/main/java/pt/ua/imodec/util/GifSequenceWriter.java b/src/main/java/pt/ua/imodec/util/GifSequenceWriter.java new file mode 100644 index 0000000..c92019e --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/GifSequenceWriter.java @@ -0,0 +1,193 @@ +package pt.ua.imodec.util; + +// +// GifSequenceWriter.java +// +// Created by Elliot Kroo on 2009-04-25. +// @ https://gist.github.com/jesuino/528703e7b1974d857b36 +// +// This work is licensed under the Creative Commons Attribution 3.0 Unported +// License. To view a copy of this license, visit +// http://creativecommons.org/licenses/by/3.0/ or send a letter to Creative +// Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. + + +import javax.imageio.*; +import javax.imageio.metadata.*; +import javax.imageio.stream.*; +import java.awt.image.*; +import java.io.*; +import java.util.Iterator; + +public class GifSequenceWriter { + protected ImageWriter gifWriter; + protected ImageWriteParam imageWriteParam; + protected IIOMetadata imageMetaData; + + /** + * Creates a new GifSequenceWriter + * + * @param outputStream the ImageOutputStream to be written to + * @param imageType one of the imageTypes specified in BufferedImage + * @param timeBetweenFramesMS the time between frames in miliseconds + * @param loopContinuously wether the gif should loop repeatedly + * @throws IIOException if no gif ImageWriters are found + * + * @author Elliot Kroo (elliot[at]kroo[dot]net) + */ + public GifSequenceWriter( + ImageOutputStream outputStream, + int imageType, + int timeBetweenFramesMS, + boolean loopContinuously) throws IOException { + // my method to create a writer + gifWriter = getWriter(); + imageWriteParam = gifWriter.getDefaultWriteParam(); + ImageTypeSpecifier imageTypeSpecifier = + ImageTypeSpecifier.createFromBufferedImageType(imageType); + + imageMetaData = + gifWriter.getDefaultImageMetadata(imageTypeSpecifier, + imageWriteParam); + + String metaFormatName = imageMetaData.getNativeMetadataFormatName(); + + IIOMetadataNode root = (IIOMetadataNode) + imageMetaData.getAsTree(metaFormatName); + + IIOMetadataNode graphicsControlExtensionNode = getNode( + root, + "GraphicControlExtension"); + + graphicsControlExtensionNode.setAttribute("disposalMethod", "none"); + graphicsControlExtensionNode.setAttribute("userInputFlag", "FALSE"); + graphicsControlExtensionNode.setAttribute( + "transparentColorFlag", + "FALSE"); + graphicsControlExtensionNode.setAttribute( + "delayTime", + Integer.toString(timeBetweenFramesMS / 10)); + graphicsControlExtensionNode.setAttribute( + "transparentColorIndex", + "0"); + + IIOMetadataNode commentsNode = getNode(root, "CommentExtensions"); + commentsNode.setAttribute("CommentExtension", "Created by MAH"); + + IIOMetadataNode appEntensionsNode = getNode( + root, + "ApplicationExtensions"); + + IIOMetadataNode child = new IIOMetadataNode("ApplicationExtension"); + + child.setAttribute("applicationID", "NETSCAPE"); + child.setAttribute("authenticationCode", "2.0"); + + int loop = loopContinuously ? 0 : 1; + + child.setUserObject(new byte[]{ 0x1, (byte) (loop & 0xFF), (byte) + (0)}); + appEntensionsNode.appendChild(child); + + imageMetaData.setFromTree(metaFormatName, root); + + gifWriter.setOutput(outputStream); + + gifWriter.prepareWriteSequence(null); + } + + public void writeToSequence(RenderedImage img) throws IOException { + gifWriter.writeToSequence( + new IIOImage( + img, + null, + imageMetaData), + imageWriteParam); + } + + /** + * Close this GifSequenceWriter object. This does not close the underlying + * stream, just finishes off the GIF. + */ + public void close() throws IOException { + gifWriter.endWriteSequence(); + } + + /** + * Returns the first available GIF ImageWriter using + * ImageIO.getImageWritersBySuffix("gif"). + * + * @return a GIF ImageWriter object + * @throws IIOException if no GIF image writers are returned + */ + private static ImageWriter getWriter() throws IIOException { + Iterator iter = ImageIO.getImageWritersBySuffix("gif"); + if(!iter.hasNext()) { + throw new IIOException("No GIF Image Writers Exist"); + } else { + return iter.next(); + } + } + + /** + * Returns an existing child node, or creates and returns a new child node (if + * the requested node does not exist). + * + * @param rootNode the IIOMetadataNode to search for the child node. + * @param nodeName the name of the child node. + * + * @return the child node, if found or a new node created with the given name. + */ + private static IIOMetadataNode getNode( + IIOMetadataNode rootNode, + String nodeName) { + int nNodes = rootNode.getLength(); + for (int i = 0; i < nNodes; i++) { + if (rootNode.item(i).getNodeName().compareToIgnoreCase(nodeName) + == 0) { + return((IIOMetadataNode) rootNode.item(i)); + } + } + IIOMetadataNode node = new IIOMetadataNode(nodeName); + rootNode.appendChild(node); + return(node); + } + + /** + public GifSequenceWriter( + BufferedOutputStream outputStream, + int imageType, + int timeBetweenFramesMS, + boolean loopContinuously) { + + */ + + public static void main(String[] args) throws Exception { + if (args.length > 1) { + // grab the output image type from the first image in the sequence + BufferedImage firstImage = ImageIO.read(new File(args[0])); + + // create a new BufferedOutputStream with the last argument + ImageOutputStream output = + new FileImageOutputStream(new File(args[args.length - 1])); + + // create a gif sequence with the type of the first image, 1 second + // between frames, which loops continuously + GifSequenceWriter writer = + new GifSequenceWriter(output, firstImage.getType(), 1, false); + + // write out the first image to our sequence... + writer.writeToSequence(firstImage); + for(int i=1; iSource. - * - * @param inputStream Stream with the dicom data - * @return The buffered image - * @throws IOException if the image format is not DICOM or another IO issue occurred - */ - public static BufferedImage loadDicomImage(DicomInputStream inputStream) throws IOException { - try (ImageInputStream imageInputStream = ImageIO.createImageInputStream(inputStream)) { - ImageReader reader = getImageReader("DICOM"); - ImageReadParam param = reader.getDefaultReadParam(); - reader.setInput(imageInputStream, false); - BufferedImage image = reader.read(0, param); - if (image == null) - throw new NullPointerException("Error reading dicom image!"); - return image; - } - } - - public static BufferedImage loadDicomImage(DicomObject dicomObject) throws IOException { - - String tmpDicomFileName = String.format("/tmp/imodec/ImageUtils/%s.dcm", - dicomObject.getString(Tag.SOPInstanceUID)); - File tmpDicomFile = new File(tmpDicomFileName); - tmpDicomFile.deleteOnExit(); + public static BufferedImage loadDicomImage(DicomObject dicomObject, int frame) throws IOException { - if (!DicomUtils.saveDicomFile(dicomObject, tmpDicomFile, true)) { - logger.warn("Hash collision in creating the tmp file. Overwriting..."); - } + File tmpDicomFile = DicomUtils.saveDicomFile(dicomObject, true); DicomInputStream dicomInputStream = new DicomInputStream(tmpDicomFile); - return loadDicomImage(dicomInputStream); + return DicomUtils.loadDicomImage(dicomInputStream, frame); } - public static ImageWriter getImageWriter(String formatName) { - - Iterator imageWriterIterator = ImageIO.getImageWritersByFormatName(formatName); - - if (!imageWriterIterator.hasNext()) - throw new NullPointerException(String.format("Format '%s' is not supported by ImageIO!", formatName)); - - return imageWriterIterator.next(); - + public static Iterator loadDicomImageIterator(DicomInputStream dicomInputStream) throws IOException { + + File file = new File(ImodecPluginSet.TMP_DIR_PATH + "/loadIteratorTmp.dcm"); + file.deleteOnExit(); + FileUtils.copyInputStreamToFile(dicomInputStream, file); + + DicomObject meta = DicomUtils.readNonPixelData(new DicomInputStream(file)); + + return new Iterator() { + + private int i = 0; + private final int frames = meta.getInt(Tag.NumberOfFrames); + private final String transferSyntax = meta.getString(Tag.TransferSyntaxUID); + private final Format format = Arrays.stream((Format[]) NewFormat.values()) + .filter(format -> format.getTransferSyntax().uid().equals(transferSyntax)) + .findFirst() + .orElse(Native.UNCHANGED); + + @Override + public boolean hasNext() { + return i + 1 < frames; + } + + @Override + public BufferedImage next() { + try { + if (format instanceof NewFormat) { + try (DicomInputStream inputStream = new DicomInputStream(file)) { + return DicomUtils.loadDicomEncodedFrame(inputStream, i, (NewFormat) format); + } + } + return DicomUtils.loadDicomImage(new DicomInputStream(file), i++); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + }; } public static ImageReader getImageReader(String formatName) { @@ -83,36 +81,4 @@ public static ImageReader getImageReader(String formatName) { } - public static void encodeDicomObject(DicomObject dicomObject, NewFormat chosenFormat) throws IOException { - - logger.info("Encoding with recent formats..."); - - if (!OSValidator.validate()) - throw new IllegalStateException( - String.format("Unsupported OS: '%s' for Imodec storage plugin", System.getProperty("os.name")) - ); - - int rawImageByteSize = dicomObject.getBytes(Tag.PixelData).length; - - // Change dicom object from uncompressed to JPEG XL, WebP or AVIF format - BufferedImage dicomImage = loadDicomImage(dicomObject); - - String tmpFileName = String.format("/tmp/imodec/ImageUtils/%d.png", dicomImage.hashCode()); - File tmpImageFile = new File(tmpFileName); - tmpImageFile.deleteOnExit(); - - ImageIO.write(dicomImage, "png", tmpImageFile); - - byte[] bitstream = NewFormatsCodecs.encodePNGFile(tmpImageFile, chosenFormat); - int compressedImageByteSize = bitstream.length; - - // Adding the new data into the dicom object - dicomObject.putBytes(Tag.PixelData, VR.OB, bitstream); - // Change parameters other than pixel-data (lossy compression, transfer syntax, ...) - dicomObject.putString(Tag.TransferSyntaxUID, VR.UI, chosenFormat.getTransferSyntax().uid()); - dicomObject.putString(Tag.LossyImageCompression, VR.CS, "01"); - dicomObject.putString(Tag.LossyImageCompressionRatio, VR.DS, String.valueOf(rawImageByteSize / compressedImageByteSize)); - dicomObject.putString(Tag.LossyImageCompressionMethod, VR.CS, chosenFormat.getMethod()); - - } } diff --git a/src/main/java/pt/ua/imodec/util/MiscUtils.java b/src/main/java/pt/ua/imodec/util/MiscUtils.java index ffef04f..91b40d3 100644 --- a/src/main/java/pt/ua/imodec/util/MiscUtils.java +++ b/src/main/java/pt/ua/imodec/util/MiscUtils.java @@ -1,12 +1,28 @@ package pt.ua.imodec.util; +import org.dcm4che2.data.DicomObject; +import org.dcm4che2.data.Tag; +import org.dcm4che2.data.TransferSyntax; +import org.dcm4che2.io.DicomInputStream; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import org.yaml.snakeyaml.Yaml; +import pt.ua.imodec.ImodecPluginSet; +import pt.ua.imodec.util.formats.NewFormat; +import pt.ua.imodec.webservice.ImodecJettyPlugin; -import java.io.File; -import java.io.IOException; -import java.io.WriteAbortedException; +import javax.imageio.ImageIO; +import javax.imageio.stream.FileImageOutputStream; +import javax.imageio.stream.ImageOutputStream; +import java.awt.image.BufferedImage; +import java.io.*; +import java.nio.file.FileAlreadyExistsException; +import java.nio.file.Files; +import java.nio.file.NoSuchFileException; +import java.time.Instant; +import java.util.*; import java.util.function.Supplier; +import java.util.stream.Collectors; public class MiscUtils { @@ -42,4 +58,166 @@ public static void sleepWhile(Supplier booleanSupplier) { } } } + + public static Map getOptions(String formatId) { + Map> options = getOptions(); + + return options.get(formatId); + } + + public static Map> getOptions() { + Yaml yaml = new Yaml(); + + InputStream inputStream = MiscUtils.class + .getClassLoader() + .getResourceAsStream("encoding-options.yaml"); + + return yaml.load(inputStream); + } + + public static Number gracefulCast(Number number, Class toType) { + if (Float.class.equals(toType)) + return number.floatValue(); + else if (Double.class.equals(toType)) + return number.doubleValue(); + else if (Integer.class.equals(toType)) + return number.intValue(); + else if (Short.class.equals(toType)) + return number.shortValue(); + else if (Byte.class.equals(toType)) + return number.byteValue(); + logger.error("Invalid type to cast to!"); + return number; + } + + /** + * + * @param bos Large output stream + * @return Input stream + */ + public static InputStream getInputStreamFromLarge(ByteArrayOutputStream bos) throws IOException { + File file = new File( + String.format("%s/blobs_%s.tmp", ImodecPluginSet.TMP_DIR_PATH, Date.from(Instant.now()))); + + if (!file.getParentFile().exists() && !file.getParentFile().mkdir()) + throw new IOException("Could not create dir: " + ImodecPluginSet.TMP_DIR_PATH); + + if (!file.createNewFile()) + throw new FileAlreadyExistsException("File -> "+file.getName()); + + FileOutputStream fileOutputStream = new FileOutputStream(file); + bos.writeTo(fileOutputStream); + InputStream inputStream = Files.newInputStream(file.toPath()); + + file.deleteOnExit(); + + return inputStream; + } + + /** + * + * @param dicomInputStream + * @param dicomObject is not expected to contain data, only the meta information + * @return + * @throws IOException + */ + public static InputStream extractImageInputStream(DicomInputStream dicomInputStream, DicomObject dicomObject) throws IOException { + + + List newFormatList = Arrays.asList(NewFormat.values()); + + List newFormatListTsUids = newFormatList + .stream() + .map(NewFormat::getTransferSyntax) + .map(TransferSyntax::uid) + .collect(Collectors.toList() + ); + + BufferedImage dicomImage; + String tsUID = dicomObject.getString(Tag.TransferSyntaxUID); + boolean isMultiframe = dicomObject.getInt(Tag.NumberOfFrames) > 1; + + if (newFormatListTsUids.contains(tsUID)) {// Case recent formats + // Parse format uid into format id + NewFormat chosenFormat = newFormatList + .stream() + .filter(newFormat -> newFormat.getTransferSyntax().uid().equals(tsUID)) + .findFirst() + .orElseThrow(UnsupportedEncodingException::new); + + if (!isMultiframe) + dicomImage = NewFormatsCodecs.decodeByteStream( + dicomObject.getBytes(Tag.PixelData), chosenFormat + ); + else + dicomImage = NewFormatsCodecs.decodeByteStream( + dicomObject.get(Tag.PixelData).getFragment(0), chosenFormat + ); + } else if (!isMultiframe) { + dicomImage = DicomUtils.loadDicomImage(dicomInputStream, 0); + } else { + + Iterator frameIterator = ImageUtils.loadDicomImageIterator(dicomInputStream); + + File gif = saveToGif(frameIterator, dicomObject.getString(Tag.SOPInstanceUID) + + "-" + tsUID); + + return Files.newInputStream(gif.toPath()); + + } + + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + ImageIO.write(dicomImage, "png", baos); + + return new ByteArrayInputStream(baos.toByteArray()); + } + + /** + * + * @param frameIterator iterator with the images + * @param gifFileBaseName Name of the resulting gif file (w/o the .gif) + * @throws IOException + */ + public static File saveToGif(Iterator frameIterator, String gifFileBaseName) throws IOException { + + String resourcesUriPath = ImodecJettyPlugin.RESOURCES_URI.getPath(); + File prefixDir = new File(resourcesUriPath); + if (!prefixDir.exists() && !prefixDir.mkdirs()) + throw new NoSuchFileException("Could not create directory: " + prefixDir); + prefixDir.deleteOnExit(); + + File gif = new File(prefixDir + "/" + gifFileBaseName + ".gif"); + if (gif.exists()) + return gif; + + if (!gif.exists() && !gif.createNewFile()) + throw new AssertionError("Unexpected error!"); + gif.deleteOnExit(); + + if (!frameIterator.hasNext()) + throw new NullPointerException("No frames to iterate over!"); + BufferedImage firstFrame = frameIterator.next(); + + ImageOutputStream fileOutputStream = new FileImageOutputStream(gif); + + GifSequenceWriter writer = new GifSequenceWriter(fileOutputStream, + firstFrame.getType(), 50, true); + + writer.writeToSequence(firstFrame); + while (frameIterator.hasNext()) + writer.writeToSequence(frameIterator.next()); + + writer.close(); + fileOutputStream.close(); + + return gif; + } + +// public static String readHtml(File template) { +// try (FileInputStream fileInputStream = new FileInputStream(template)) { +// fileInputStream.wr +// } catch (IOException e) { +// throw new RuntimeException(e); +// } +// } } diff --git a/src/main/java/pt/ua/imodec/util/NewFormat.java b/src/main/java/pt/ua/imodec/util/NewFormat.java deleted file mode 100644 index e8d3b8c..0000000 --- a/src/main/java/pt/ua/imodec/util/NewFormat.java +++ /dev/null @@ -1,50 +0,0 @@ -package pt.ua.imodec.util; - -import org.dcm4che2.data.TransferSyntax; - -public enum NewFormat { - - JPEG_XL(NewFormatsTS.JPEG_XL_TS, "ISO_18181", "jxl"), - WEBP(NewFormatsTS.WEBP_TS, "webp", "webp"), - AVIF(NewFormatsTS.AVIF, "avif", "avif"); - - private final TransferSyntax transferSyntax; - private final String method, fileExtension; - - NewFormat(TransferSyntax transferSyntax, String method, String fileExtension) { - this.transferSyntax = transferSyntax; - this.method = method; - this.fileExtension = fileExtension; - } - - public TransferSyntax getTransferSyntax() { - return transferSyntax; - } - - public String getMethod() { - return method; - } - - public String getFileExtension() { - return fileExtension; - } - - private static class NewFormatsTS { - - private static final String AVIF_TS_UID = "1.2.826.0.1.3680043.2.682.104.3"; - - private static final String JPEG_XL_TS_UID = "1.2.826.0.1.3680043.2.682.104.1"; - - private static final String WEBP_TS_UID = "1.2.826.0.1.3680043.2.682.104.2"; - - public static final TransferSyntax JPEG_XL_TS = new TransferSyntax(JPEG_XL_TS_UID, - false, false, true, false); - - public static final TransferSyntax WEBP_TS = new TransferSyntax(WEBP_TS_UID, - false, false, true, false); - - public static final TransferSyntax AVIF = new TransferSyntax(AVIF_TS_UID, - false, false, true, false); - - } -} diff --git a/src/main/java/pt/ua/imodec/util/NewFormatsCodecs.java b/src/main/java/pt/ua/imodec/util/NewFormatsCodecs.java index e63ef01..ad57130 100644 --- a/src/main/java/pt/ua/imodec/util/NewFormatsCodecs.java +++ b/src/main/java/pt/ua/imodec/util/NewFormatsCodecs.java @@ -1,5 +1,10 @@ package pt.ua.imodec.util; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import pt.ua.imodec.ImodecPluginSet; +import pt.ua.imodec.util.formats.NewFormat; + import javax.imageio.ImageIO; import java.awt.image.BufferedImage; import java.io.File; @@ -7,85 +12,89 @@ import java.nio.file.Files; import java.nio.file.Paths; import java.util.Arrays; +import java.util.HashMap; public class NewFormatsCodecs { + private static final Logger logger = LoggerFactory.getLogger(NewFormatsCodecs.class); private static final String losslessFormat = "png"; - public static byte[] encodePNGFile(File tmpImageFile, NewFormat chosenFormat) throws IOException { - - // Input validation - assert tmpImageFile.exists(): String.format("PNG file to encode '%s' does not exist.", tmpImageFile); - - String tmpFilePath = tmpImageFile.getAbsolutePath(); - String formatExtension; + public static byte[] encodePNGFile(File pngFile, NewFormat chosenFormat, HashMap options) + throws IOException { - switch (chosenFormat) { - case JPEG_XL: - - formatExtension = "jxl"; + // Input validation + assert pngFile.exists(): String.format("PNG file to encode '%s' does not exist.", pngFile); - return encode(tmpFilePath, formatExtension); + String formatExtension = chosenFormat.getFileExtension(); + String pngFilePath = pngFile.getAbsolutePath(); + String encodedFileName = pngFilePath.replace("png", formatExtension); - case WEBP: + String encodingCommand = getCodecCommand(pngFilePath, encodedFileName, formatExtension, + true, options); - formatExtension = "webp"; + execute(encodingCommand); - return encode(tmpFilePath, formatExtension); - case AVIF: + File encodedImageFile = new File(encodedFileName); + encodedImageFile.deleteOnExit(); - formatExtension = "avif"; + return Files.readAllBytes(encodedImageFile.toPath()); - return encode(tmpFilePath, formatExtension); - default: - throw new IllegalStateException("Unexpected format!"); - } } - private static byte[] encode(String inputFilePath, String formatExtension) throws IOException { - - String encodedFileName = inputFilePath.replace("png", formatExtension); - - String encodingCommand = getCodecCommand(inputFilePath, encodedFileName, formatExtension, true); - + private static void execute(String encodingCommand) throws IOException { // Execute the command and wait for it to finish Process compression = Runtime.getRuntime().exec(encodingCommand); try { compression.waitFor(); } catch (InterruptedException ignored) {} - File encodedImageFile = new File(encodedFileName); - - encodedImageFile.deleteOnExit(); - - return Files.readAllBytes(encodedImageFile.toPath()); + if (compression.exitValue() != 0) { + logger.error("Problem executing process: '{}' failed unexpectedly with error '{}'.\n" + + "Error code: %s", encodingCommand, compression.exitValue()); + throw new AssertionError("Unexpected error"); + } } private static String getCodecCommand(String inputPath, String outputPath, - String formatExtension, boolean encoding) { + String formatExtension, boolean encoding, HashMap options) { assert new File(inputPath).exists(): "Input file does not exist!"; assert !(new File(outputPath).exists()): "Output file already exists!"; - char codecId; - - if (encoding) - codecId = 'c'; - else - codecId = 'd'; - - String outputPrefix = "-o"; - - if (formatExtension.equals(NewFormat.JPEG_XL.getFileExtension())) - outputPrefix = ""; - - if (!encoding && formatExtension.equals(NewFormat.AVIF.getFileExtension())) - return String.format("avif_decode %s %s", inputPath, outputPath); + switch (formatExtension) { + case "jxl": + if (encoding) { + Number distance = options.getOrDefault("distance", NewFormat.JPEG_XL + .getQualityParamValue()); + Number effort = options.getOrDefault("effort", NewFormat.JPEG_XL.getSpeedParamValue()); + + return String.format("cjxl %s %s --effort=%s --distance=%s", + inputPath, outputPath, effort, distance); + } + return String.format("djxl %s %s", inputPath, outputPath); + case "avif": + if (encoding) { + Number quality = options.getOrDefault("quality", NewFormat.AVIF.getQualityParamValue()), + speed = options.getOrDefault("speed", NewFormat.AVIF.getSpeedParamValue()); + return String.format("cavif -o %s --quality %s --speed %s %s", outputPath, quality, speed, + inputPath); + } + return String.format("avif_decode -f %s %s", inputPath, outputPath); + case "webp": + if (encoding) { + Number quality = options.getOrDefault("quality", NewFormat.WEBP.getQualityParamValue()), + speed = options.getOrDefault("speed", NewFormat.WEBP.getSpeedParamValue()); + return String.format("cwebp -q %s -m %s %s -o %s", quality, speed, inputPath, outputPath); + } + return String.format("dwebp %s -o %s", inputPath, outputPath); + default: + throw new IllegalArgumentException("Format is not valid!"); + } - return String.format("%c%s %s %s %s", codecId, formatExtension, inputPath, outputPrefix, outputPath); } public static BufferedImage decodeByteStream(byte[] bitstream, NewFormat chosenFormat) throws IOException { - String encodedFileName = String.format("/tmp/imodec/%s.%s", Arrays.hashCode(bitstream), chosenFormat.getFileExtension()); + String encodedFileName = String.format("%s/%s.%s", ImodecPluginSet.TMP_DIR_PATH, + Arrays.hashCode(bitstream), chosenFormat.getFileExtension()); Files.write(Paths.get(encodedFileName), bitstream); return decode(encodedFileName, chosenFormat.getFileExtension()); } @@ -93,16 +102,16 @@ public static BufferedImage decodeByteStream(byte[] bitstream, NewFormat chosenF private static BufferedImage decode(String inputFilePath, String formatExtension) throws IOException { String decodedFileName = inputFilePath.replace(formatExtension, losslessFormat); + File decodedImageFile = new File(decodedFileName); + if (decodedImageFile.exists()) + return ImageIO.read(decodedImageFile); - String decodingCommand = getCodecCommand(inputFilePath, decodedFileName, formatExtension, false); + String decodingCommand = getCodecCommand(inputFilePath, decodedFileName, formatExtension, false, + new HashMap<>()); - Process decompression = Runtime.getRuntime().exec(decodingCommand); + execute(decodingCommand); - try { - decompression.waitFor(); - } catch (InterruptedException ignored) {} - - File decodedImageFile = new File(decodedFileName); + decodedImageFile.deleteOnExit(); return ImageIO.read(decodedImageFile); diff --git a/src/main/java/pt/ua/imodec/util/formats/Format.java b/src/main/java/pt/ua/imodec/util/formats/Format.java new file mode 100644 index 0000000..f9b654d --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/formats/Format.java @@ -0,0 +1,10 @@ +package pt.ua.imodec.util.formats; + +import org.dcm4che2.data.TransferSyntax; + +public interface Format { + + TransferSyntax getTransferSyntax(); + + String getId(); +} diff --git a/src/main/java/pt/ua/imodec/util/formats/Native.java b/src/main/java/pt/ua/imodec/util/formats/Native.java new file mode 100644 index 0000000..1e98a91 --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/formats/Native.java @@ -0,0 +1,29 @@ +package pt.ua.imodec.util.formats; + +import org.dcm4che2.data.TransferSyntax; + +public enum Native implements Format { + + IMPLICIT_VR_BIG_ENDIAN(TransferSyntax.ImplicitVRBigEndian, "ibe"), + EXPLICIT_VR_BIG_ENDIAN(TransferSyntax.ExplicitVRBigEndian, "ebe"), + IMPLICIT_VR_LITTLE_ENDIAN(TransferSyntax.ImplicitVRLittleEndian, "ile"), + EXPLICIT_VR_LITTLE_ENDIAN(TransferSyntax.ExplicitVRLittleEndian, "ele"), + UNCHANGED(null, "keep"); + + private final TransferSyntax transferSyntax; + private final String id; + + Native(TransferSyntax transferSyntax, String id) { + this.transferSyntax = transferSyntax; + this.id = id; + } + + @Override + public TransferSyntax getTransferSyntax() { + return transferSyntax; + } + + public String getId() { + return id; + } +} diff --git a/src/main/java/pt/ua/imodec/util/formats/NewFormat.java b/src/main/java/pt/ua/imodec/util/formats/NewFormat.java new file mode 100644 index 0000000..7d66e2f --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/formats/NewFormat.java @@ -0,0 +1,95 @@ +package pt.ua.imodec.util.formats; + +import org.dcm4che2.data.TransferSyntax; +import pt.ua.imodec.util.MiscUtils; + +public enum NewFormat implements Format { + + JPEG_XL(NewFormatsTS.JPEG_XL_TS, "ISO_18181", + "jxl", "distance", "effort", Float.class), + WEBP(NewFormatsTS.WEBP_TS, "webp", + "webp", "quality", "speed", Byte.class), + AVIF(NewFormatsTS.AVIF_TS, "avif", + "avif", "quality", "speed", Byte.class); + + private final TransferSyntax transferSyntax; + private final String method, fileExtension, id, qualityParamName, speedParamName; + + private Number qualityParamValue, speedParamValue; + private final Class qualityParamType; + + NewFormat(TransferSyntax transferSyntax, String method, String fileExtension, String qualityParamName, + String speedParamName, Class qualityValueType) { + this.transferSyntax = transferSyntax; + this.method = method; + this.fileExtension = fileExtension; + this.id = fileExtension; + this.qualityParamName = qualityParamName; + Number number = MiscUtils.getOptions(fileExtension).get(qualityParamName); + this.qualityParamValue = MiscUtils.gracefulCast(number, qualityValueType); + this.qualityParamType = qualityValueType; + this.speedParamName = speedParamName; + this.speedParamValue = MiscUtils.getOptions(fileExtension).get(speedParamName).byteValue(); + } + + @Override + public TransferSyntax getTransferSyntax() { + return transferSyntax; + } + + public String getMethod() { + return method; + } + + public String getFileExtension() { + return fileExtension; + } + + @Override + public String getId() { + return id; + } + + public Number getQualityParamValue() { + return qualityParamValue; + } + + public Number getSpeedParamValue() { + return speedParamValue; + } + + public void setQualityParamValue(Number qualityParamValue) { + this.qualityParamValue = MiscUtils.gracefulCast(qualityParamValue, qualityParamType); + } + + public void setSpeedParamValue(Number speedParamValue) { + this.speedParamValue = speedParamValue.byteValue(); + } + + public String getQualityParamName() { + return qualityParamName; + } + + public String getSpeedParamName() { + return speedParamName; + } + + private static class NewFormatsTS { + + private static final String AVIF_TS_UID = "1.2.826.0.1.3680043.2.682.104.3"; + + private static final String JPEG_XL_TS_UID = "1.2.826.0.1.3680043.2.682.104.1"; + + private static final String WEBP_TS_UID = "1.2.826.0.1.3680043.2.682.104.2"; + + public static final TransferSyntax JPEG_XL_TS = new TransferSyntax(JPEG_XL_TS_UID, + false, false, true, false); + + public static final TransferSyntax WEBP_TS = new TransferSyntax(WEBP_TS_UID, + false, false, true, false); + + public static final TransferSyntax AVIF_TS = new TransferSyntax(AVIF_TS_UID, + false, false, true, false); + + } +} diff --git a/src/main/java/pt/ua/imodec/util/validators/EncodeabilityValidator.java b/src/main/java/pt/ua/imodec/util/validators/EncodeabilityValidator.java new file mode 100644 index 0000000..d4e741a --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/validators/EncodeabilityValidator.java @@ -0,0 +1,29 @@ +package pt.ua.imodec.util.validators; + +import org.dcm4che2.data.DicomObject; +import org.dcm4che2.data.Tag; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class EncodeabilityValidator implements Validator { + + private static final Logger logger = LoggerFactory.getLogger(EncodeabilityValidator.class); + + public static boolean validate(DicomObject dicomObject) { + if (dicomObject.contains(Tag.LossyImageCompression) + && dicomObject.getString(Tag.LossyImageCompression).equals("01")) { + logger.error("Lossy image compression has already been subjected, thus it cannot be re-applied. " + + "Aborting..."); + return false; + } + if (dicomObject.contains(Tag.AllowLossyCompression) + && dicomObject.getString(Tag.AllowLossyCompression).equals("NO")) { + logger.error("Lossy compression is not allowed to be applied in this dicom object " + + "(AllowLossyCompression field is set false)"); + return false; + } + + return true; + } + +} diff --git a/src/main/java/pt/ua/imodec/util/validators/OSValidator.java b/src/main/java/pt/ua/imodec/util/validators/OSValidator.java index 1b1ed97..8772a9f 100644 --- a/src/main/java/pt/ua/imodec/util/validators/OSValidator.java +++ b/src/main/java/pt/ua/imodec/util/validators/OSValidator.java @@ -1,6 +1,6 @@ package pt.ua.imodec.util.validators; -public class OSValidator { +public class OSValidator implements Validator { private static final String OS = System.getProperty("os.name").toLowerCase(); diff --git a/src/main/java/pt/ua/imodec/util/validators/Validator.java b/src/main/java/pt/ua/imodec/util/validators/Validator.java new file mode 100644 index 0000000..0cfc109 --- /dev/null +++ b/src/main/java/pt/ua/imodec/util/validators/Validator.java @@ -0,0 +1,9 @@ +package pt.ua.imodec.util.validators; + +public interface Validator { + + static boolean validate(Object... args) { + return false; + } + +} diff --git a/src/main/java/pt/ua/imodec/webservice/ImodecJettyPlugin.java b/src/main/java/pt/ua/imodec/webservice/ImodecJettyPlugin.java index 72ded6c..140d4e8 100644 --- a/src/main/java/pt/ua/imodec/webservice/ImodecJettyPlugin.java +++ b/src/main/java/pt/ua/imodec/webservice/ImodecJettyPlugin.java @@ -3,6 +3,7 @@ import org.eclipse.jetty.server.handler.HandlerList; import org.eclipse.jetty.servlet.ServletContextHandler; import org.eclipse.jetty.servlet.ServletHolder; +import org.eclipse.jetty.util.resource.Resource; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import pt.ua.dicoogle.sdk.JettyPluginInterface; @@ -10,6 +11,10 @@ import pt.ua.dicoogle.sdk.core.PlatformCommunicatorInterface; import pt.ua.dicoogle.sdk.settings.ConfigurationHolder; +import java.io.File; +import java.io.IOException; +import java.net.URI; + /** * Jetty Servlet plugin, based on bioinformatics-ua/dicoogle-plugin-sample * @@ -19,6 +24,8 @@ public class ImodecJettyPlugin implements JettyPluginInterface, PlatformCommunicatorInterface { private static final Logger logger = LoggerFactory.getLogger(ImodecJettyPlugin.class); + public static final URI RESOURCES_URI = new File("tmp/").toURI(); + public static final String CONTEXT_PATH = "/imodec"; private final ImodecJettyWebService webService; private boolean enabled; private ConfigurationHolder settings; @@ -77,7 +84,16 @@ public void setSettings(ConfigurationHolder settings) { public HandlerList getJettyHandlers() { ServletContextHandler handler = new ServletContextHandler(); - handler.setContextPath("/imodec"); + handler.setContextPath(CONTEXT_PATH); + + logger.debug("Creating base resource..."); + try { + Resource resource = Resource.newResource(RESOURCES_URI); + handler.setBaseResource(resource); + } catch (IOException e) { + throw new RuntimeException(e); + } + handler.addServlet(new ServletHolder(this.webService), "/view"); // Example: path to access this servlet? example below // GET http://localhost:8080/imodec/view?param=value diff --git a/src/main/java/pt/ua/imodec/webservice/ImodecJettyWebService.java b/src/main/java/pt/ua/imodec/webservice/ImodecJettyWebService.java index 18ece39..02fc8a9 100644 --- a/src/main/java/pt/ua/imodec/webservice/ImodecJettyWebService.java +++ b/src/main/java/pt/ua/imodec/webservice/ImodecJettyWebService.java @@ -2,28 +2,27 @@ import org.dcm4che2.data.DicomObject; import org.dcm4che2.data.Tag; -import org.dcm4che2.data.TransferSyntax; import org.dcm4che2.io.DicomInputStream; -import org.dcm4che2.io.DicomOutputStream; +import org.eclipse.jetty.util.resource.Resource; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import pt.ua.dicoogle.sdk.QueryInterface; import pt.ua.dicoogle.sdk.StorageInputStream; import pt.ua.dicoogle.sdk.core.DicooglePlatformInterface; import pt.ua.dicoogle.sdk.core.PlatformCommunicatorInterface; -import pt.ua.dicoogle.sdk.datastructs.SearchResult; import pt.ua.imodec.storage.ImodecStoragePlugin; -import pt.ua.imodec.util.ImageUtils; -import pt.ua.imodec.util.NewFormat; -import pt.ua.imodec.util.NewFormatsCodecs; +import pt.ua.imodec.util.*; +import pt.ua.imodec.util.formats.Format; +import pt.ua.imodec.util.formats.Native; +import pt.ua.imodec.util.formats.NewFormat; -import javax.imageio.ImageIO; +import javax.servlet.ServletContext; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.awt.image.BufferedImage; import java.io.*; import java.net.URI; +import java.net.URISyntaxException; import java.util.*; import java.util.stream.Collectors; @@ -36,6 +35,8 @@ public class ImodecJettyWebService extends HttpServlet implements PlatformCommunicatorInterface { private static final Logger logger = LoggerFactory.getLogger(ImodecJettyWebService.class); private static final String sopInstanceUIDParameterName = "siuid"; + private static final String transferSyntaxUIDParameterName = "tsuid"; + private static final String formatIdParameterName = "codec"; public static final String storageScheme = new ImodecStoragePlugin().getScheme(); private DicooglePlatformInterface platform; @@ -54,45 +55,71 @@ public ImodecJettyWebService() {} @Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException { + ServletContext servletContext = request.getServletContext(); + DicomInputStream dicomInputStream = extractRequestedDicomFromStorage(request); - DicomObject dicomObject = extractRequestedDicomFromStorage(request).readDicomObject(); + DicomObject dicomObject = DicomUtils.readNonPixelData(extractRequestedDicomFromStorage(request)); - response.setContentType("image/png"); + boolean isMultiFrame = dicomObject.getInt(Tag.NumberOfFrames) > 1; - List newFormatList = Arrays.asList(NewFormat.values()); + String tsUID = dicomObject.getString(Tag.TransferSyntaxUID); + Format chosenFormat = Arrays.stream(((Format[]) NewFormat.values())) + .filter(format -> format.getTransferSyntax().uid().equals(tsUID)) + .findFirst() + .orElse(Native.UNCHANGED); - List newFormatListTsUids = newFormatList - .stream() - .map(NewFormat::getTransferSyntax) - .map(TransferSyntax::uid) - .collect(Collectors.toList() - ); + if (isMultiFrame) { + if (chosenFormat.equals(Native.UNCHANGED)) { + response.setContentType("text/html;charset=utf-8"); - BufferedImage dicomImage; - String tsUID = dicomObject.getString(Tag.TransferSyntaxUID); + Iterator frameIterator = ImageUtils.loadDicomImageIterator(dicomInputStream); + + File gif = MiscUtils.saveToGif(frameIterator, dicomObject.getString(Tag.SOPInstanceUID) + "-" + tsUID); + + PrintWriter printWriter = response.getWriter(); + + Optional gifResource = Optional.ofNullable( + Resource.newResource(servletContext.getResource("/" + gif.getName())) + ); + + printWriter.println(""); + printWriter.println(""); + printWriter.println(""); + if (gifResource.isPresent() && gifResource.get().exists()) { + printWriter.printf("\"Image\n", gif.getName()); // FIXME: 16/09/22 Image always fails + printWriter.printf("Copy and paste this link to view the image locally." + + "\n", gifResource.get().getURL()); + } else { + logger.error("Gif file was not found!"); + printWriter.println("Error code 500! Multi-frame image does not exist!"); + } + printWriter.println(""); - if (newFormatListTsUids.contains(tsUID)) {// Case recent formats - // Parse format uid into format id - NewFormat chosenFormat = newFormatList - .stream() - .filter(newFormat -> newFormat.getTransferSyntax().uid().equals(tsUID)) - .findFirst().get(); + return; + } else if (chosenFormat instanceof NewFormat) { + response.setContentType("text/html;charset=utf-8"); - dicomImage = NewFormatsCodecs.decodeByteStream( - dicomObject.getBytes(Tag.PixelData), chosenFormat - ); + PrintWriter printWriter = response.getWriter(); + + String output = "Displaying multi-frame dicom encoded with recent formats is not yet supported!"; + + printWriter.printf("%s\n", output); + logger.warn(output); + + return; + } } - else - dicomImage = ImageUtils.loadDicomImage(dicomInputStream); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - ImageIO.write(dicomImage, "png", baos); + dicomObject = extractRequestedDicomFromStorage(request).readDicomObject(); + + response.setContentType("image/png"); + + InputStream inputStream = MiscUtils.extractImageInputStream(dicomInputStream, dicomObject); - ByteArrayInputStream inputStream = new ByteArrayInputStream(baos.toByteArray()); BufferedOutputStream servletOutputStream = new BufferedOutputStream(response.getOutputStream()); int ch; - while ((ch=inputStream.read()) != -1) + while ((ch = inputStream.read()) != -1) servletOutputStream.write(ch); servletOutputStream.close(); @@ -100,37 +127,66 @@ protected void doGet(HttpServletRequest request, HttpServletResponse response) t } + /** + * Analyzes the request and returns the associated dicom object + * + * @param request + * @return + * @throws IOException + */ private DicomInputStream extractRequestedDicomFromStorage(HttpServletRequest request) throws IOException { - String sopInstanceUID = request.getParameter(sopInstanceUIDParameterName); + String sopInstanceUID = request.getParameter(sopInstanceUIDParameterName), + transferSyntaxUID = request.getParameter(transferSyntaxUIDParameterName), + codecId = request.getParameter(formatIdParameterName); - URI uri = URI.create(storageScheme + "://" + sopInstanceUID); + if (transferSyntaxUID == null && codecId != null) { + Format format = Arrays.stream((Format[]) (NewFormat.values())) + .filter(newFormat -> newFormat.getId().equals(codecId)) + .findFirst() + .orElse(Native.UNCHANGED); - Iterable files = this.platform.getStorageForSchema(uri).at(uri); + transferSyntaxUID = format.getTransferSyntax().uid(); + } else if (transferSyntaxUID == null) + transferSyntaxUID = findNativeVersionTS(sopInstanceUID); - Iterator storageInputStreamIterator = files.iterator(); + URI uri = URI.create(storageScheme + "://" + sopInstanceUID + "/" + transferSyntaxUID); - InputStream inputStream = storageInputStreamIterator.next().getInputStream(); - - return new DicomInputStream(inputStream); + return getDicomInputStream(uri); } - private DicomInputStream extractRequestedDicomFromIndex(HttpServletRequest request) throws IOException { - - String SOPInstanceUID = request.getParameter("uid"); - - QueryInterface dimProvider = this.platform.getQueryProviderByName("lucene", true); - Iterator results = dimProvider.query("SOPInstanceUID:" + SOPInstanceUID).iterator(); + private String findNativeVersionTS(String sopInstanceUID) { + String transferSyntaxUID; + Collection nativeUIDs = Arrays.stream(Native.values()) + .filter(aNative -> aNative.getTransferSyntax() != null) + .collect(Collectors.toList()); + + String imodecStorageScheme = new ImodecStoragePlugin().getScheme(); + ImodecStoragePlugin imodecStorage = ((ImodecStoragePlugin) this.platform.getStorageForSchema(imodecStorageScheme)); + + transferSyntaxUID = nativeUIDs.stream().filter(aNative -> { + try { + return imodecStorage.containsURI( + new URI(String.format("%s://%s/%s", + imodecStorageScheme, sopInstanceUID, aNative.getTransferSyntax().uid()) + ) + ); + } catch (URISyntaxException e) { + throw new RuntimeException(e); + } + }).findFirst() + .orElseThrow(NoSuchElementException::new) // TODO: 19/09/22 Give a better error message using logger perhaps + .getTransferSyntax().uid(); + return transferSyntaxUID; + } - if (!results.hasNext()) - throw new NoSuchElementException("No dicom file found!"); + private DicomInputStream getDicomInputStream(URI uri) throws IOException { + Iterable files = this.platform.getStorageForSchema(uri).at(uri); - SearchResult res = results.next(); - URI uri = res.getURI(); + Iterator storageInputStreamIterator = files.iterator(); - if (uri == null) - throw new NullPointerException("Null uri for the requested dicom object!"); + InputStream inputStream = storageInputStreamIterator.next().getInputStream(); - return new DicomInputStream(new File(uri)); + return new DicomInputStream(inputStream); } @Override diff --git a/src/main/resources/encoding-options.yaml b/src/main/resources/encoding-options.yaml new file mode 100644 index 0000000..251d854 --- /dev/null +++ b/src/main/resources/encoding-options.yaml @@ -0,0 +1,9 @@ +jxl: + distance: 1.0 + effort: 7 +avif: + quality: 90 + speed: 4 +webp: + quality: 90 + speed: 4 \ No newline at end of file diff --git a/src/main/resources/viewer-template.html b/src/main/resources/viewer-template.html new file mode 100644 index 0000000..5cfcda2 --- /dev/null +++ b/src/main/resources/viewer-template.html @@ -0,0 +1,10 @@ + + + + + Viewer + + +${} + + \ No newline at end of file From 2b6395c6f1fe4e46dc909d01597c47a56b608dab Mon Sep 17 00:00:00 2001 From: Almeida-a Date: Mon, 26 Sep 2022 12:14:39 +0100 Subject: [PATCH 3/4] Bump to 0.1.0-b0 (#27) * Update with main (#26) * Hotfix: bump the maven tag to v0.1.0-a0 (#8) * Tweaked pom (changed artifactID) * Main class added * Reverted artifact ID * Removed name * Tweaked pom * Re-added maven.compiler tags * Tried multiple mvn compiler versions * Removed java 8 from CI (for now) * Re-added java 8 to compiler and ci options * Re-added java 8 to compiler and ci options * Re-added java 8 to ci options * Added test/ci branch to ci scope * Restructured code into the plugin set form * Updated dicoogle version pointer * Template Storage plugin * Implemented interface contract methods * Registered storage plugin * Changed context path name Plus, added some comments: * explaining getJettyHandlers * providing a path example to access a servlet * Fixed target version to 3.0.5 * Fixed target version to 3.0.5 * Simple image viewer (by "get" operation) of uncompressed dicom images * Image store encoding with recent formats (#4) * #3 Introduced ability to transcode the pixel-data into newer (compressed) formats * #3 Complemented transcoding process by adding lossy compression attributes as well as transfer syntax Also fixed problem of file creation in MiscUtils * #3 Added format decoding functions * #3 Fixed multiple bugs * Thread waits for compression processes to finish * Files generated by the encoders are now deleted upon exit * Added logic for detecting and avoiding duplicate file storing (although, this may already be enforced by dicoogle itself) * #3 decode bugfix: thread waits for decompression subprocess to finish * Viewer for encoded dicom images. (#6) * #5 decode feature: provided method to retrieve buffered image from a dicom object with an encoded pixel-data * #5 image viewer: finalized the feature Also: other minor fixes and tweaks. * #5 image viewer: Added configuration to choose the format Also: fixed the "davif" command decode - using avif_decode (rust) instead * Bump maven version tag to v0.1.0-a0 * Bump to 0.1.0-a1 (#23) * Allow storing pixel-data in native format (#17) * #14 Feature: Allow for native transfer syntaxes By default or if specified * #14 Refactor: organized the enums into a single package & cherry-picked the readme from main & changed the 'same' keyword that is to keep the transfer syntax to 'keep' * #14 Feature: Enable saving all transfer syntaxes at once By all TS, it is meant all new formats' TSs, as well as the native version (as long the original format is not lossy) * #14 Feature: Enable viewing any available TS version of a dicom object * Using the tsuid or codec parameter - if left unspecified, then use native form * TS - Transfer Syntax(es) * #14 Docs: Updates on the viewer http request parameters * Fixed readme merge mistake * Docs: Add table of contents to the readme * #14 Fix: Adding 'all' to the codec config tag * Create CODE_OF_CONDUCT.md (#18) * Update issue templates * Added contributing section to readme * Allow defining encoding parameters (#19) * #15 Feature: quality and speed parameters available However, still beyond reach of configuration by the user Also: error handling for subprocesses * #15 fix: merge mistake * #15 refactor: inlined encode function into encodePNGFile Also, encoding options are now set to default * #15 Feature: Default encoding options are now extracted from yml From encoding-options.yaml Also, quality value number type of each encoder is now specified in NewFormat enum Also, removed unused code * #15 Feature: Allow defining quality and speed encoding parameters Configuration is performed at DicoogleDir/Plugins/settings/imodec-plugin-set.xml * #15 Feature: Warning when encoding parameters are invalid * Support for storing multi-frame images* (#22) * #11 Refactor: Paved way to start developing support for multi-frame dicom objects * #11 Refactor: Memory optimization when codec -> all Optimization is to use iterator, for each dicom object clone, instead of array - more memory efficient Also, fixed bug when getting uri (was not getting the correct TS sometimes) Also, changed some dependencies * #11 Feature: Store multi-frame images * #11 Finished feature*: Viewing raw multi-frame images as gif * Workaround - Could not find a way yet to display animated images. Thus, server can only view them locally by a file:// link * I marked the source of the above problem with a 'fixme 16-09-22' tag * #11 Finished feature: Multi-frame store operation * #11 [Bugfix] Some old features that were broken By the multi-frame support feature * Code cleanup * [Refactor] Moves functions pertaining utility to *Utils.java * #11 [Bugfix] could not read single frame avif * #11 [README] Informed that `all` does not work with multi-frames storing operation * Update maven version tag to v0.1.0-b0 --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index d869438..b208621 100644 --- a/pom.xml +++ b/pom.xml @@ -6,7 +6,7 @@ pt.ua.ieeta imodec-dicoogle-plugin-set imodec-dicoogle-plugin-set - 0.1.0-a0 + 0.1.0-b0 jar @@ -139,4 +139,4 @@ - \ No newline at end of file + From 6cb6302db7030fb1fc96c6c3b6ada602367a80a9 Mon Sep 17 00:00:00 2001 From: Almeida-a Date: Mon, 26 Sep 2022 18:50:49 +0100 Subject: [PATCH 4/4] Hotfix 1 to v0.1.0-b0 (#28) * Update with main (#26) * Hotfix: bump the maven tag to v0.1.0-a0 (#8) * Tweaked pom (changed artifactID) * Main class added * Reverted artifact ID * Removed name * Tweaked pom * Re-added maven.compiler tags * Tried multiple mvn compiler versions * Removed java 8 from CI (for now) * Re-added java 8 to compiler and ci options * Re-added java 8 to compiler and ci options * Re-added java 8 to ci options * Added test/ci branch to ci scope * Restructured code into the plugin set form * Updated dicoogle version pointer * Template Storage plugin * Implemented interface contract methods * Registered storage plugin * Changed context path name Plus, added some comments: * explaining getJettyHandlers * providing a path example to access a servlet * Fixed target version to 3.0.5 * Fixed target version to 3.0.5 * Simple image viewer (by "get" operation) of uncompressed dicom images * Image store encoding with recent formats (#4) * #3 Introduced ability to transcode the pixel-data into newer (compressed) formats * #3 Complemented transcoding process by adding lossy compression attributes as well as transfer syntax Also fixed problem of file creation in MiscUtils * #3 Added format decoding functions * #3 Fixed multiple bugs * Thread waits for compression processes to finish * Files generated by the encoders are now deleted upon exit * Added logic for detecting and avoiding duplicate file storing (although, this may already be enforced by dicoogle itself) * #3 decode bugfix: thread waits for decompression subprocess to finish * Viewer for encoded dicom images. (#6) * #5 decode feature: provided method to retrieve buffered image from a dicom object with an encoded pixel-data * #5 image viewer: finalized the feature Also: other minor fixes and tweaks. * #5 image viewer: Added configuration to choose the format Also: fixed the "davif" command decode - using avif_decode (rust) instead * Bump maven version tag to v0.1.0-a0 * Bump to 0.1.0-a1 (#23) * Allow storing pixel-data in native format (#17) * #14 Feature: Allow for native transfer syntaxes By default or if specified * #14 Refactor: organized the enums into a single package & cherry-picked the readme from main & changed the 'same' keyword that is to keep the transfer syntax to 'keep' * #14 Feature: Enable saving all transfer syntaxes at once By all TS, it is meant all new formats' TSs, as well as the native version (as long the original format is not lossy) * #14 Feature: Enable viewing any available TS version of a dicom object * Using the tsuid or codec parameter - if left unspecified, then use native form * TS - Transfer Syntax(es) * #14 Docs: Updates on the viewer http request parameters * Fixed readme merge mistake * Docs: Add table of contents to the readme * #14 Fix: Adding 'all' to the codec config tag * Create CODE_OF_CONDUCT.md (#18) * Update issue templates * Added contributing section to readme * Allow defining encoding parameters (#19) * #15 Feature: quality and speed parameters available However, still beyond reach of configuration by the user Also: error handling for subprocesses * #15 fix: merge mistake * #15 refactor: inlined encode function into encodePNGFile Also, encoding options are now set to default * #15 Feature: Default encoding options are now extracted from yml From encoding-options.yaml Also, quality value number type of each encoder is now specified in NewFormat enum Also, removed unused code * #15 Feature: Allow defining quality and speed encoding parameters Configuration is performed at DicoogleDir/Plugins/settings/imodec-plugin-set.xml * #15 Feature: Warning when encoding parameters are invalid * Support for storing multi-frame images* (#22) * #11 Refactor: Paved way to start developing support for multi-frame dicom objects * #11 Refactor: Memory optimization when codec -> all Optimization is to use iterator, for each dicom object clone, instead of array - more memory efficient Also, fixed bug when getting uri (was not getting the correct TS sometimes) Also, changed some dependencies * #11 Feature: Store multi-frame images * #11 Finished feature*: Viewing raw multi-frame images as gif * Workaround - Could not find a way yet to display animated images. Thus, server can only view them locally by a file:// link * I marked the source of the above problem with a 'fixme 16-09-22' tag * #11 Finished feature: Multi-frame store operation * #11 [Bugfix] Some old features that were broken By the multi-frame support feature * Code cleanup * [Refactor] Moves functions pertaining utility to *Utils.java * #11 [Bugfix] could not read single frame avif * #11 [README] Informed that `all` does not work with multi-frames storing operation * Update maven version tag to v0.1.0-b0 * [Chore] (pom.xml): Fixed some build warning problems * [Tests]: Hidden home directory & Fixed a test The test fixed was an unintentionally skipped one, which I didn't previously notice. That skip was caused by a (now fixed) bug, and the test itself had errors (shadowed by the skip). Those errors were fixed. --- pom.xml | 4 ++-- .../pt/ua/imodec/util/DicomUtilsTest.java | 24 +++++++++---------- .../pt/ua/imodec/util/ImageUtilsTest.java | 4 ++-- .../java/pt/ua/imodec/util/MiscUtilsTest.java | 9 +++++-- .../java/pt/ua/imodec/util/TestCommons.java | 6 +++++ 5 files changed, 29 insertions(+), 18 deletions(-) create mode 100644 src/test/java/pt/ua/imodec/util/TestCommons.java diff --git a/pom.xml b/pom.xml index b208621..1dc7609 100644 --- a/pom.xml +++ b/pom.xml @@ -10,7 +10,7 @@ jar - UTF-8 + UTF-8 3.0.0 @@ -46,7 +46,7 @@ org.junit.jupiter junit-jupiter - RELEASE + 5.9.0 test diff --git a/src/test/java/pt/ua/imodec/util/DicomUtilsTest.java b/src/test/java/pt/ua/imodec/util/DicomUtilsTest.java index 481ed5c..a7d1035 100644 --- a/src/test/java/pt/ua/imodec/util/DicomUtilsTest.java +++ b/src/test/java/pt/ua/imodec/util/DicomUtilsTest.java @@ -33,8 +33,6 @@ class DicomUtilsTest { - public static final String DICOM_DATASET_DIR = "/home/archy/ic-encoders-eval/images/dataset_dicom/"; - /** * Overload of isReadable(DicomObject). * Also checks if the dicom object can be read from the dicomInputStream without triggering OutOfMemoryError @@ -132,6 +130,8 @@ private static Optional tryLoadDicom(File dicomFile) { } private static File tryToFindDatasetDir(String datasetDir) { + datasetDir = datasetDir.replace("~", System.getProperty("user.home")); + Path datasetDirPath = Paths.get(datasetDir); File datasetDirFile = datasetDirPath.toFile(); Assumptions.assumeTrue(datasetDirFile.exists() && datasetDirFile.isDirectory()); @@ -149,7 +149,7 @@ static boolean isReadable(DicomObject dicomObject) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void saveDicomFile(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir); @@ -184,7 +184,7 @@ void saveDicomFile(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void readNonPixelData(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir); @@ -207,7 +207,7 @@ void readNonPixelData(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) @Disabled void loadDicomEncodedFrame(String datasetDir) { @@ -228,7 +228,7 @@ void loadDicomEncodedFrame(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void loadDicomImage(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir); @@ -269,7 +269,7 @@ void loadDicomImage(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void encodeSingleFrameDicomObject(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir); @@ -314,7 +314,7 @@ void encodeSingleFrameDicomObject(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void encodeMultiFrameDicomObject(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir, 2, false, true); @@ -354,7 +354,7 @@ void encodeMultiFrameDicomObject(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void encodeIteratorDicomObjectWithAllTs(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir); @@ -400,7 +400,7 @@ void encodeIteratorDicomObjectWithAllTs(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) @Disabled void encodeIteratorDicomInputStreamWithAllTs(String datasetDir) { @@ -449,7 +449,7 @@ void encodeIteratorDicomInputStreamWithAllTs(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {DICOM_DATASET_DIR}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void writeDicomObjectToTmpFile(String datasetDir) { Iterator dicomIterator = getDicomDatasetFiles(datasetDir); @@ -466,7 +466,7 @@ void writeDicomObjectToTmpFile(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {DICOM_DATASET_DIR}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void testGetDicomDatasetFiles(String datasetDir) { byte iterationsLimit = 3; diff --git a/src/test/java/pt/ua/imodec/util/ImageUtilsTest.java b/src/test/java/pt/ua/imodec/util/ImageUtilsTest.java index 48ae8bd..fb1dda3 100644 --- a/src/test/java/pt/ua/imodec/util/ImageUtilsTest.java +++ b/src/test/java/pt/ua/imodec/util/ImageUtilsTest.java @@ -52,7 +52,7 @@ void getImageReader() { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void loadDicomImage(String datasetDir) { Iterator dicomIterator = DicomUtilsTest.getDicomDatasetFiles(datasetDir); @@ -83,7 +83,7 @@ void loadDicomImage(String datasetDir) { } @ParameterizedTest - @ValueSource(strings = {"/home/archy/ic-encoders-eval/images/dataset_dicom/"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void loadDicomImageIterator(String datasetDir) { Iterator dicomIterator = DicomUtilsTest.getDicomDatasetFiles(datasetDir); diff --git a/src/test/java/pt/ua/imodec/util/MiscUtilsTest.java b/src/test/java/pt/ua/imodec/util/MiscUtilsTest.java index 3e79e43..ca5d532 100644 --- a/src/test/java/pt/ua/imodec/util/MiscUtilsTest.java +++ b/src/test/java/pt/ua/imodec/util/MiscUtilsTest.java @@ -141,7 +141,7 @@ void getInputStreamFromLarge() throws IOException { } @ParameterizedTest - @ValueSource(strings = {"~/ic-encoders-eval/images/dataset-dicom"}) + @ValueSource(strings = {TestCommons.DICOM_DATASET_DIR}) void cloneInputStream(String datasetDir) { Iterator filesDataset = DicomUtilsTest.getDicomDatasetFiles(datasetDir); @@ -152,7 +152,12 @@ void cloneInputStream(String datasetDir) { InputStream inputStream2 = assertDoesNotThrow(() -> FileUtils.openInputStream(file)); InputStream inputStream3 = assertDoesNotThrow(() -> MiscUtils.cloneInputStream(inputStream1)); - assertEquals(inputStream2, inputStream3); + byte[] data2 = new byte[assertDoesNotThrow(inputStream2::available)]; + assertDoesNotThrow(() -> inputStream2.read(data2)); + byte[] data3 = new byte[assertDoesNotThrow(inputStream3::available)]; + assertDoesNotThrow(() -> inputStream3.read(data3)); + + assertArrayEquals(data2, data3); assertDoesNotThrow(inputStream1::close); assertDoesNotThrow(inputStream2::close); diff --git a/src/test/java/pt/ua/imodec/util/TestCommons.java b/src/test/java/pt/ua/imodec/util/TestCommons.java new file mode 100644 index 0000000..2025653 --- /dev/null +++ b/src/test/java/pt/ua/imodec/util/TestCommons.java @@ -0,0 +1,6 @@ +package pt.ua.imodec.util; + +public class TestCommons { + + static final String DICOM_DATASET_DIR = "~/ic-encoders-eval/images/dataset_dicom/"; +}