Mergeback 1.5.2 to 1.6.0 (#3214)

* Mergeback 1.4.4 to 1.5.0 (#2745) * Update MAPI version (#2730) * Update anomaly ov inference task * Update reqs in exportable code * Fix one more place of conversion of anomaly map * Update dependency for exportable code (#2732) * Fix unsupported dtype in ov graph constant converter --------- Co-authored-by: Vladislav Sovrasov <sovrasov.vlad@gmail.com> * Mergeback 1.4.0 to 1.5.0 (#2826) * Update MAPI version (#2730) * Update dependency for exportable code (#2732) * Filter invalid polygon shapes (#2795) --------- Co-authored-by: Vladislav Sovrasov <sovrasov.vlad@gmail.com> Co-authored-by: Eugene Liu <eugene.liu@intel.com> * Mergeback changes for 1.4.5 to releases/1.5.0 (#2871) * Update MAPI version (#2730) * Update anomaly ov inference task * Fix one more place of conversion of anomaly map * Update dependency for exportable code (#2732) * Fix unsupported dtype in ov graph constant converter * Filter invalid polygon shapes (#2795) * fix segmentation test * Update for releases 1.4.5 (#2837) * 🐞 Bugfix: Set reverse_input_channels to True in OpenVINO models (#2848) * Set reverse_input_channels to True * Remove dependency of protobuf (#2851) * Update release notes and changelog for 1.4.5 (#2853) --------- Co-authored-by: Vladislav Sovrasov <sovrasov.vlad@gmail.com> Co-authored-by: Eugene Liu <eugene.liu@intel.com> Co-authored-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * Update version string to 1.5.1 (#2874) update version string to 1.5.1 * [Release 1.5] Fix label order for h-cls (#2921) Fix h-cls label order * Revert MRCNN resize to state from 1.4 (#2922) * Update openvino.txt (#2923) * Remove polygon clipping code (#2926) * Fix default memcache size to 100MB (#2960) * Fix default memcache size to 100MB * Deal with empty annotation of 'No Object' label images --------- Signed-off-by: Songki Choi <songki.choi@intel.com> * Hotfix default memcache size to 100MB (#2990) * Fix default memcache size to 100MB * Fix memcache size max limit to 10GB * Update CHANGELOG.md * Hotfix: fix wrong domain in tiling rotated detection (#3141) * domain=rotated_detection in tile_pipeline * update tile pipeline * Update otx version to 1.5.2 (#3174) * update otx version to 1.5.2 * Update CHANGELOG.md --------- Co-authored-by: Vladislav Sovrasov <sovrasov.vlad@gmail.com> Co-authored-by: Eugene Liu <eugene.liu@intel.com> Co-authored-by: Ashwin Vaidya <ashwin.vaidya@intel.com> Co-authored-by: Galina Zalesskaya <galina.zalesskaya@intel.com> Co-authored-by: Emily Chun <emily.chun@intel.com> Co-authored-by: Songki Choi <songki.choi@intel.com>
openvinotoolkit · Mar 29, 2024 · 620dfd1 · 620dfd1
1 parent 48f84b1
commit 620dfd1
Show file tree

Hide file tree

Showing 21 changed files with 286 additions and 51 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,8 @@ All notable changes to this project will be documented in this file.
 
 ## \[unreleased\]
 
+## \[v1.6.0\]
+
 ### New features
 
 - Add zero-shot visual prompting (<https://github.com/openvinotoolkit/training_extensions/pull/2616>, <https://github.com/openvinotoolkit/training_extensions/pull/2706>, <https://github.com/openvinotoolkit/training_extensions/pull/2753>)
@@ -13,6 +15,18 @@ All notable changes to this project will be documented in this file.
 
 - Upgrade OpenVINO to 2023.3 (<https://github.com/openvinotoolkit/training_extensions/pull/2879>)
 - Automate performance benchmark (<https://github.com/openvinotoolkit/training_extensions/pull/2742>)
+- Bump ONNX version to 1.16.0 to resolve CVE-2022-25882 (<https://github.com/openvinotoolkit/training_extensions/pull/3205>)
+
+## \[v1.5.2\]
+
+> **NOTES**
+>
+> OpenVINO™ Training Extension v1.5.2 does not include the latest functional and security updates. OpenVINO™ Training Extension v2.0.0 is targeted to be released in April 2024 and will include additional functional and security updates. Customers should update to the latest version as it becomes available.
+
+### Bug fixes
+
+- Remove polygon clipping code (<https://github.com/openvinotoolkit/training_extensions/pull/2926>)
+- Hotfix default memcache size to 100MB (<https://github.com/openvinotoolkit/training_extensions/pull/2990>)
 
 ## \[v1.5.0\]
 
@@ -49,6 +63,17 @@ All notable changes to this project will be documented in this file.
 - OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch>=1.13.1
 - NNCF QAT optimization is disabled for MaskRCNN models due to CUDA runtime error in ROIAlign kernel on torch==2.0.1
 
+## \[v1.4.5\]
+
+### Bug fixes
+
+- Filter invalid polygon shapes (<https://github.com/openvinotoolkit/training_extensions/pull/2795>)
+- 🐞 Bugfix: Set reverse*input_channels to True in OpenVINO models (<https://github.com/openvinotoolkit/training_extensions/pull/2848>*
+
+### Misc
+
+- Remove dependency of protobuf to omit vulnerability issue (<https://github.com/openvinotoolkit/training_extensions/pull/2851>)
+
 ## \[v1.4.4\]
 
 ### Enhancements
@@ -324,7 +349,7 @@ All notable changes to this project will be documented in this file.
 
 ## \[v1.0.0\]
 
-> _**NOTES**_
+> **NOTES**
 >
 > OpenVINO™ Training Extensions which version 1.0.0 has been updated to include functional and security updates. Users should update to the latest version.
 
@@ -352,7 +377,7 @@ All notable changes to this project will be documented in this file.
 
 ## \[v0.5.0\]
 
-> _**NOTES**_
+> **NOTES**
 >
 > OpenVINO Training Extension which version is equal or older then v0.5.0 does not include the latest functional and security updates. OTE Version 1.0.0 is targeted to be released in February 2023 and will include additional functional and security updates. Customers should update to the latest version as it becomes available.
 

diff --git a/docs/source/guide/release_notes/index.rst b/docs/source/guide/release_notes/index.rst
@@ -4,6 +4,13 @@ Releases
 .. toctree::
    :maxdepth: 1
 
+v1.5.2 (1Q24)
+-------------
+
+- Remove polygon clipping code
+- Hotfix default memcache size to 100MB
+
+
 v1.5.0 (4Q23)
 -------------
 
@@ -25,6 +32,12 @@ v1.5.0 (4Q23)
 - Support torch==2.0.1
 - Set "Auto" as default input size mode
 
+v1.4.5 (1Q24)
+-------------
+
+- Filter invalid polygon shapes
+- Fix a bug to set reverse_input_channels for OpenVINO models
+- Remove unreferenced dependency of protobuf
 
 v1.4.4 (4Q23)
 -------------

diff --git a/requirements/base.txt b/requirements/base.txt
@@ -2,7 +2,6 @@
 # Base Algo Requirements.                                                     #
 natsort==8.1.*
 prettytable==3.9.*
-protobuf==3.20.*
 pyyaml
 datumaro==1.6.0rc1
 psutil==5.9.*

diff --git a/src/otx/algorithms/action/configs/classification/configuration.yaml b/src/otx/algorithms/action/configs/classification/configuration.yaml
@@ -290,7 +290,7 @@ algo_backend:
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool
-    max_value: 9223372036854775807
+    max_value: 10000000000
     min_value: 0
     type: INTEGER
     ui_rules:

diff --git a/src/otx/algorithms/action/configs/detection/configuration.yaml b/src/otx/algorithms/action/configs/detection/configuration.yaml
@@ -290,7 +290,7 @@ algo_backend:
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool
-    max_value: 9223372036854775807
+    max_value: 10000000000
     min_value: 0
     type: INTEGER
     ui_rules:

diff --git a/src/otx/algorithms/anomaly/tasks/inference.py b/src/otx/algorithms/anomaly/tasks/inference.py
@@ -357,7 +357,7 @@ def _add_metadata_to_ir(self, model_file: str, export_type: ExportType) -> None:
         if "min" in metadata and "max" in metadata:
             extra_model_data[("model_info", "normalization_scale")] = metadata["max"] - metadata["min"]
 
-        extra_model_data[("model_info", "reverse_input_channels")] = False
+        extra_model_data[("model_info", "reverse_input_channels")] = True  # convert BGR to RGB
         extra_model_data[("model_info", "model_type")] = "AnomalyDetection"
 
         labels = []

diff --git a/src/otx/algorithms/anomaly/tasks/openvino.py b/src/otx/algorithms/anomaly/tasks/openvino.py
@@ -453,7 +453,7 @@ def _metadata_in_ir_format(self) -> Dict[Tuple[str, str], Union[str, int, float,
         if "min" in metadata and "max" in metadata:
             extra_model_data[("model_info", "normalization_scale")] = metadata["max"] - metadata["min"]
 
-        extra_model_data[("model_info", "reverse_input_channels")] = False
+        extra_model_data[("model_info", "reverse_input_channels")] = True  # convert BGR to RGB
         extra_model_data[("model_info", "model_type")] = "AnomalyDetection"
         extra_model_data[("model_info", "labels")] = "Normal Anomaly"
         return extra_model_data

diff --git a/src/otx/algorithms/classification/adapters/openvino/task.py b/src/otx/algorithms/classification/adapters/openvino/task.py
@@ -127,7 +127,11 @@ def __init__(
 
         self.model = Model.create_model(model_adapter, "Classification", self.configuration, preload=True)
 
-        self.converter = ClassificationToAnnotationConverter(self.label_schema)
+        if self.model.hierarchical:
+            hierarchical_cls_heads_info = self.model.hierarchical_info["cls_heads_info"]
+        else:
+            hierarchical_cls_heads_info = None
+        self.converter = ClassificationToAnnotationConverter(self.label_schema, hierarchical_cls_heads_info)
         self.callback_exceptions: List[Exception] = []
         self.model.inference_adapter.set_callback(self._async_callback)
 

diff --git a/src/otx/algorithms/classification/configs/configuration.yaml b/src/otx/algorithms/classification/configs/configuration.yaml
@@ -441,11 +441,11 @@ algo_backend:
     warning: null
   mem_cache_size:
     affects_outcome_of: TRAINING
-    default_value: 1000000000
+    default_value: 100000000
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool
-    max_value: 9223372036854775807
+    max_value: 10000000000
     min_value: 0
     type: INTEGER
     ui_rules:

diff --git a/src/otx/algorithms/classification/utils/cls_utils.py b/src/otx/algorithms/classification/utils/cls_utils.py
@@ -18,8 +18,9 @@
 
 import json
 from operator import itemgetter
-from typing import Any, Dict
+from typing import Any, Dict, List
 
+from otx.api.entities.label import LabelEntity
 from otx.api.entities.label_schema import LabelSchemaEntity
 from otx.api.serialization.label_mapper import LabelSchemaMapper
 
@@ -51,8 +52,8 @@ def get_multihead_class_info(label_schema: LabelSchemaEntity):  # pylint: disabl
     for j, group in enumerate(single_label_groups):
         class_to_idx[group[0]] = (len(exclusive_groups), j)
 
-    all_labels = label_schema.get_labels(include_empty=False)
-    label_to_idx = {lbl.name: i for i, lbl in enumerate(all_labels)}
+    # Idx of label corresponds to model output
+    label_to_idx = {lbl: i for i, lbl in enumerate(class_to_idx.keys())}
 
     mixed_cls_heads_info = {
         "num_multiclass_heads": len(exclusive_groups),
@@ -104,9 +105,13 @@ def get_cls_model_api_configuration(label_schema: LabelSchemaEntity, inference_c
     mapi_config[("model_info", "hierarchical")] = str(inference_config["hierarchical"])
     mapi_config[("model_info", "output_raw_scores")] = str(True)
 
+    label_entities = label_schema.get_labels(include_empty=False)
+    if inference_config["hierarchical"]:
+        label_entities = get_hierarchical_label_list(inference_config["multihead_class_info"], label_entities)
+
     all_labels = ""
     all_label_ids = ""
-    for lbl in label_schema.get_labels(include_empty=False):
+    for lbl in label_entities:
         all_labels += lbl.name.replace(" ", "_") + " "
         all_label_ids += f"{lbl.id_} "
 
@@ -123,22 +128,16 @@ def get_cls_model_api_configuration(label_schema: LabelSchemaEntity, inference_c
     return mapi_config
 
 
-def get_hierarchical_label_list(hierarchical_info, labels):
+def get_hierarchical_label_list(hierarchical_cls_heads_info: Dict, labels: List) -> List[LabelEntity]:
     """Return hierarchical labels list which is adjusted to model outputs classes."""
+
+    # Create the list of Label Entities (took from "labels")
+    # corresponding to names and order in "label_to_idx"
+    label_to_idx = hierarchical_cls_heads_info["label_to_idx"]
     hierarchical_labels = []
-    for head_idx in range(hierarchical_info["num_multiclass_heads"]):
-        logits_begin, logits_end = hierarchical_info["head_idx_to_logits_range"][str(head_idx)]
-        for logit in range(0, logits_end - logits_begin):
-            label_str = hierarchical_info["all_groups"][head_idx][logit]
-            label_idx = hierarchical_info["label_to_idx"][label_str]
-            hierarchical_labels.append(labels[label_idx])
-
-    if hierarchical_info["num_multilabel_classes"]:
-        logits_begin = hierarchical_info["num_single_label_classes"]
-        logits_end = len(labels)
-        for logit_idx, logit in enumerate(range(0, logits_end - logits_begin)):
-            label_str_idx = hierarchical_info["num_multiclass_heads"] + logit_idx
-            label_str = hierarchical_info["all_groups"][label_str_idx][0]
-            label_idx = hierarchical_info["label_to_idx"][label_str]
-            hierarchical_labels.append(labels[label_idx])
+    for label_str, _ in label_to_idx.items():
+        for label_entity in labels:
+            if label_entity.name == label_str:
+                hierarchical_labels.append(label_entity)
+                break
     return hierarchical_labels
diff --git a/src/otx/algorithms/common/adapters/mmcv/pipelines/load_image_from_otx_dataset.py b/src/otx/algorithms/common/adapters/mmcv/pipelines/load_image_from_otx_dataset.py
@@ -204,6 +204,8 @@ def __call__(self, results: Dict[str, Any]) -> Dict[str, Any]:
             return cached_results
         results = self._load_img(results)
         results = self._load_ann_if_any(results)
+        if results is None:
+            return None
         results.pop("dataset_item", None)  # Prevent deepcopy or caching
         results = self._resize_img_ann_if_any(results)
         self._save_cache(results)

diff --git a/src/otx/algorithms/detection/configs/detection/configuration.yaml b/src/otx/algorithms/detection/configs/detection/configuration.yaml
@@ -371,11 +371,11 @@ algo_backend:
     warning: null
   mem_cache_size:
     affects_outcome_of: TRAINING
-    default_value: 1000000000
+    default_value: 100000000
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool
-    max_value: 9223372036854775807
+    max_value: 10000000000
     min_value: 0
     type: INTEGER
     ui_rules:

diff --git a/src/otx/algorithms/detection/configs/instance_segmentation/configuration.yaml b/src/otx/algorithms/detection/configs/instance_segmentation/configuration.yaml
@@ -371,11 +371,11 @@ algo_backend:
     warning: null
   mem_cache_size:
     affects_outcome_of: TRAINING
-    default_value: 1000000000
+    default_value: 100000000
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool
-    max_value: 9223372036854775807
+    max_value: 10000000000
     min_value: 0
     type: INTEGER
     ui_rules:

diff --git a/src/otx/algorithms/detection/configs/rotated_detection/configuration.yaml b/src/otx/algorithms/detection/configs/rotated_detection/configuration.yaml
@@ -356,11 +356,11 @@ algo_backend:
     warning: null
   mem_cache_size:
     affects_outcome_of: TRAINING
-    default_value: 1000000000
+    default_value: 100000000
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool
-    max_value: 9223372036854775807
+    max_value: 10000000000
     min_value: 0
     type: INTEGER
     ui_rules:

diff --git a/.../algorithms/detection/configs/rotated_detection/efficientnetb2b_maskrcnn/tile_pipeline.py b/.../algorithms/detection/configs/rotated_detection/efficientnetb2b_maskrcnn/tile_pipeline.py
@@ -1,7 +1,95 @@
-"""Tiling Pipeline of EfficientNetB2B model for Instance-Seg Task."""
+"""Tiling Pipeline of EfficientNetB2B model."""
 
 # Copyright (C) 2023 Intel Corporation
 # SPDX-License-Identifier: Apache-2.0
 
+# pylint: disable=invalid-name
 
-_base_ = ["../../base/data/tiling/efficientnet_iseg_tile_pipeline.py"]
+img_size = (512, 512)
+
+tile_cfg = dict(
+    tile_size=400, min_area_ratio=0.9, overlap_ratio=0.2, iou_threshold=0.45, max_per_img=1500, filter_empty_gt=True
+)
+
+img_norm_cfg = dict(mean=(103.53, 116.28, 123.675), std=(1.0, 1.0, 1.0), to_rgb=True)
+
+train_pipeline = [
+    dict(type="Resize", img_scale=img_size, keep_ratio=False),
+    dict(type="RandomFlip", flip_ratio=0.5),
+    dict(type="Normalize", **img_norm_cfg),
+    dict(type="Pad", size_divisor=32),
+    dict(type="DefaultFormatBundle"),
+    dict(
+        type="Collect",
+        keys=["img", "gt_bboxes", "gt_labels", "gt_masks"],
+        meta_keys=[
+            "filename",
+            "ori_filename",
+            "ori_shape",
+            "img_shape",
+            "pad_shape",
+            "scale_factor",
+            "flip",
+            "flip_direction",
+            "img_norm_cfg",
+        ],
+    ),
+]
+
+test_pipeline = [
+    dict(
+        type="MultiScaleFlipAug",
+        img_scale=img_size,
+        flip=False,
+        transforms=[
+            dict(type="Resize", keep_ratio=False),
+            dict(type="RandomFlip"),
+            dict(type="Normalize", **img_norm_cfg),
+            dict(type="Pad", size_divisor=32),
+            dict(type="ImageToTensor", keys=["img"]),
+            dict(type="Collect", keys=["img"]),
+        ],
+    )
+]
+
+__dataset_type = "OTXDetDataset"
+
+train_dataset = dict(
+    type="ImageTilingDataset",
+    dataset=dict(
+        type=__dataset_type,
+        pipeline=[
+            dict(type="LoadImageFromOTXDataset", enable_memcache=True),
+            dict(type="LoadAnnotationFromOTXDataset", domain="rotated_detection", with_bbox=True, with_mask=True),
+        ],
+    ),
+    pipeline=train_pipeline,
+    **tile_cfg
+)
+
+val_dataset = dict(
+    type="ImageTilingDataset",
+    dataset=dict(
+        type=__dataset_type,
+        pipeline=[
+            dict(type="LoadImageFromOTXDataset", enable_memcache=True),
+            dict(type="LoadAnnotationFromOTXDataset", domain="rotated_detection", with_bbox=True, with_mask=True),
+        ],
+    ),
+    pipeline=test_pipeline,
+    **tile_cfg
+)
+
+test_dataset = dict(
+    type="ImageTilingDataset",
+    dataset=dict(
+        type=__dataset_type,
+        test_mode=True,
+        pipeline=[dict(type="LoadImageFromOTXDataset")],
+    ),
+    pipeline=test_pipeline,
+    **tile_cfg
+)
+
+
+data = dict(train=train_dataset, val=val_dataset, test=test_dataset)