Fix JPEG2K fused decoding (with ROI), add native tests for JP2k decoding #2692

JanuszL · 2021-02-18T13:59:32Z

adds a CPU fallback for fused (slice/crop/randomCrop) JPEG2000 images decoding
enables JPEG2000 tests for fused CPU/Mixed decoder
when nvJPEG2000 is not available for given HW fallback to CPU implementation

Signed-off-by: Janusz Lisiecki jlisiecki@nvidia.com

Why we need this PR?

Pick one, remove the rest

It fixes JPEG2K fused decoding (with ROI)
It adds native tests for JP2k decoding

What happened in this PR?

Fill relevant points, put NA otherwise. Replace anything inside []

What solution was applied:
extended tests to use JPEG2K images
adds a CPU fallback for fused (slice/crop/randomCrop) JPEG2000 images decoding
adds check if nvJPEG2000 was initialized successfully and can be used
Affected modules and functionalities:
ImageDecoder for mixed backend
tests for fused ImageDecoder (CPU, Mixed)
Key points relevant for the review:
NA
Validation and testing:
new tests are added
Documentation (including examples):
NA

JIRA TASK: []

JanuszL · 2021-02-18T19:08:51Z

!build

dali-automaton · 2021-02-18T19:10:58Z

CI MESSAGE: [2087123]: BUILD STARTED

dali-automaton · 2021-02-18T19:18:17Z

CI MESSAGE: [2087123]: BUILD FAILED

JanuszL · 2021-02-19T01:09:29Z

!build

dali-automaton · 2021-02-19T01:16:20Z

CI MESSAGE: [2088671]: BUILD STARTED

JanuszL · 2021-02-19T01:50:40Z

!build

dali-automaton · 2021-02-19T01:56:11Z

CI MESSAGE: [2088861]: BUILD STARTED

dali-automaton · 2021-02-19T03:01:39Z

CI MESSAGE: [2088671]: BUILD PASSED

dali-automaton · 2021-02-19T03:24:06Z

CI MESSAGE: [2088861]: BUILD PASSED

jantonguirao · 2021-02-19T08:33:37Z

dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h

@@ -526,7 +526,7 @@ class nvJPEGDecoder : public Operator<MixedBackend>, CachedDecoderImpl {
          data.method = DecodeMethod::NvjpegCuda;
          samples_single_.push_back(&data);
        }
-      } else if (!ParseNvjpeg2k(data, span<const uint8_t>(input_data, in_size))) {
+      } else if (crop_generator || !ParseNvjpeg2k(data, span<const uint8_t>(input_data, in_size))) {


This looks good for now. Just an idea to be discussed with the team:
In case that ROI decoding is not an option with nvjpeg2k, should we decode the whole image with nvjpeg2k and launch the GPU slice kernel to produce the result instead?

Added a comment to the schema docs,

JanuszL · 2021-02-19T11:14:42Z

!build

dali-automaton · 2021-02-19T11:21:11Z

CI MESSAGE: [2090052]: BUILD STARTED

- adds a CPU fallback for fused (slice/crop/randomCrop) JPEG2000 images decoding - enables JPEG2000 tests for fused CPU/Mixed decoder - when nvJPEG2000 is not available for given HW fallback to CPU implementation Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

jantonguirao · 2021-02-19T13:36:20Z

dali/operators/decoder/image_decoder.cc

-Please note that GPU acceleration for JPEG 2000 decoding is only available for CUDA 11.
+
+.. note::
+  JPEG 2000 decoding and extraction of regions-of-interest (ROI) in not accelerated on the GPU when


How about

JPEG 2000 region-of-interest (ROI) decoding is not accelerated on the GPU, and will use a CPU implementation regardless of the selected backend. For a GPU accelerated implementation, consider using separate ``image_decoder`` and ``crop`` operators.

jantonguirao · 2021-02-19T13:36:57Z

dali/operators/decoder/image_decoder.cc

@@ -220,7 +223,10 @@ image format, it will decode the entire image and crop the selected ROI.
 The output of the decoder is in *HWC* layout.

 Supported formats: JPG, BMP, PNG, TIFF, PNM, PPM, PGM, PBM, JPEG 2000.
-Please note that GPU acceleration for JPEG 2000 decoding is only available for CUDA 11.
+
+.. note::


JPEG 2000 region-of-interest (ROI) decoding is not accelerated on the GPU, and will use a CPU implementation regardless of the selected backend. For a GPU accelerated implementation, consider using separate ``image_decoder`` and ``random_crop`` operators.

jantonguirao · 2021-02-19T13:37:05Z

dali/operators/decoder/image_decoder.cc

@@ -273,7 +279,10 @@ image format, it will decode the entire image and crop the selected ROI.
 The output of the decoder is in the *HWC* layout.

 Supported formats: JPG, BMP, PNG, TIFF, PNM, PPM, PGM, PBM, JPEG 2000.
-Please note that GPU acceleration for JPEG 2000 decoding is only available for CUDA 11.
+
+.. note::


JPEG 2000 region-of-interest (ROI) decoding is not accelerated on the GPU, and will use a CPU implementation regardless of the selected backend. For a GPU accelerated implementation, consider using separate ``image_decoder`` and ``slice`` operators.

jantonguirao · 2021-02-19T13:42:41Z

dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h

+      // nvJPEG2k doesn't support deprecated sm while DALI does. In this case, NvJPEG2KHandle may
+      // be empty and we need to fall back to the CPU implementation
+      if (nvjpeg2k_handle_) {
+        auto nvjpeg2k_thread_id = nvjpeg2k_thread_.GetThreadIds()[0];


I feel it'd be better to get the thread id before posting the work

jantonguirao · 2021-02-19T13:43:27Z

dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h

-                                 "component 0 has a shape of {", comp.component_height, ", ",
-                                 comp.component_width, "} and component ", c, " has a shape of {",
-                                 height, ", ", width, "}"));
+    if (nvjpeg2k_handle_) {


Suggested change

if (nvjpeg2k_handle_) {

if (!nvjpeg2k_handle_)

return false;

just a suggestion

jantonguirao · 2021-02-19T13:43:58Z

dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h

+        for (uint32_t c = 1; c < image_info.num_components; ++c) {
+          NVJPEG2K_CALL(nvjpeg2kStreamGetImageComponentInfo(jpeg2k_stream, &comp, c));
+          DALI_ENFORCE(height == comp.component_height &&
+                      width == comp.component_width,


nitpick: indentation

jantonguirao · 2021-02-19T13:45:08Z

dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h

-        CacheStore(sample->file_name, output_data, shape, nvjpeg2k_cu_stream_);
-      }
-    });
+    if (nvjpeg2k_handle_) {


Similarly, I'd rather add an early return. But that might be just a personal preference so feel free to ignore

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

JanuszL · 2021-02-19T14:06:51Z

!build

dali-automaton · 2021-02-19T14:13:14Z

CI MESSAGE: [2090329]: BUILD FAILED

dali-automaton · 2021-02-19T15:29:04Z

CI MESSAGE: [2090052]: BUILD FAILED

dali-automaton · 2021-02-19T22:52:00Z

CI MESSAGE: [2090329]: BUILD STARTED

dali-automaton · 2021-02-19T22:59:32Z

CI MESSAGE: [2090052]: BUILD PASSED

dali-automaton · 2021-02-20T00:39:08Z

CI MESSAGE: [2090329]: BUILD PASSED

JanuszL force-pushed the fix_jp2k_fused_decoding branch from 6139ed4 to 9a3b0d2 Compare February 18, 2021 19:08

JanuszL force-pushed the fix_jp2k_fused_decoding branch from 9a3b0d2 to 45030ba Compare February 19, 2021 01:09

JanuszL changed the title ~~[WIP] Fix JPEG2K fused decoding~~ Fix JPEG2K fused decoding Feb 19, 2021

JanuszL changed the title ~~Fix JPEG2K fused decoding~~ Fix JPEG2K fused decoding (with ROI), add native tests for JP2k decoding Feb 19, 2021

jantonguirao self-assigned this Feb 19, 2021

jantonguirao reviewed Feb 19, 2021

View reviewed changes

jantonguirao approved these changes Feb 19, 2021

View reviewed changes

banasraf self-assigned this Feb 19, 2021

banasraf approved these changes Feb 19, 2021

View reviewed changes

JanuszL force-pushed the fix_jp2k_fused_decoding branch from 45030ba to d422eb6 Compare February 19, 2021 11:13

JanuszL force-pushed the fix_jp2k_fused_decoding branch from d422eb6 to 5b9c282 Compare February 19, 2021 11:47

jantonguirao self-requested a review February 19, 2021 12:23

jantonguirao reviewed Feb 19, 2021

View reviewed changes

Review fixes

90de0de

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

JanuszL force-pushed the fix_jp2k_fused_decoding branch from 0f3363b to 90de0de Compare February 19, 2021 14:06

jantonguirao approved these changes Feb 19, 2021

View reviewed changes

banasraf approved these changes Feb 19, 2021

View reviewed changes

JanuszL merged commit 7b9681a into NVIDIA:master Feb 20, 2021

JanuszL deleted the fix_jp2k_fused_decoding branch February 20, 2021 01:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix JPEG2K fused decoding (with ROI), add native tests for JP2k decoding #2692

Fix JPEG2K fused decoding (with ROI), add native tests for JP2k decoding #2692

JanuszL commented Feb 18, 2021 •

edited

Loading

JanuszL commented Feb 18, 2021

dali-automaton commented Feb 18, 2021

dali-automaton commented Feb 18, 2021

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

jantonguirao Feb 19, 2021

JanuszL Feb 19, 2021

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 20, 2021

Fix JPEG2K fused decoding (with ROI), add native tests for JP2k decoding #2692

Fix JPEG2K fused decoding (with ROI), add native tests for JP2k decoding #2692

Conversation

JanuszL commented Feb 18, 2021 • edited Loading

Why we need this PR?

What happened in this PR?

JanuszL commented Feb 18, 2021

dali-automaton commented Feb 18, 2021

dali-automaton commented Feb 18, 2021

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JanuszL commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 19, 2021

dali-automaton commented Feb 20, 2021

JanuszL commented Feb 18, 2021 •

edited

Loading