Feature/update version 5.1

_config.yml

@@ -27,6 +27,7 @@ navbar-links:
   Home: ""
   Docs:
     - Overview: "docs/site/index.html"
+    - Release Notes: "docs/site/Release-Notes/index.html"
     - Install: "docs/site/Install-Guide/index.html"
     - API: "docs/site/Component-API-Overview/index.html"
   Open Source:

docs/docs/Release-Notes.md

@@ -1,12 +1,149 @@
 > **NOTICE:** This software (or technical data) was produced for the U.S. Government under contract, and is subject to the Rights in Data-General Clause 52.227-14, Alt. IV (DEC 2007). Copyright 2020 The MITRE Corporation. All Rights Reserved.
 
-<!--
-# OpenMPF 5.1.0: XXX 2020
+# OpenMPF 5.1.0: November 2020
+
+<h2>Media Inspection Improvements</h2>
+
+- The Workflow Manager will now handle video files that don't have a video stream as an `AUDIO` type, and handle video files that don't have a video or audio stream as an `UNKNOWN` type. The JSON output object  contains a new `media.mediaType` field that will be set to `VIDEO`, `AUDIO`, `IMAGE`, or `UNKNOWN`.
+- The Workflow Manager now configures Tika with [custom MIME type support](https://github.com/openmpf/openmpf/blob/master/trunk/workflow-manager/src/main/resources/org/apache/tika/mime/custom-mimetypes.xml). Currently, this enables the detection of `video/vnd.dlna.mpeg-tts` and `image/jxr` MIME types.
+- If the Workflow Manager cannot use Tika to determine the media MIME type then it will fall back to using the Linux `file` command with a [custom magicfile](https://github.com/openmpf/openmpf/blob/master/trunk/workflow-manager/src/main/resources/magic/custom-magic).
+- OpenMPF now supports Apple-optimized PNGs and HEIC images. Refer to the Bug Fixes section below.
+
+<h2>EAST Text Region Detection Component Improvements</h2>
+
+- The `TEMPORARY_PADDING` property has been separated into `TEMPORARY_PADDING_X` and `TEMPORARY_PADDING_Y` so that X and Y padding can be configured independently.
+- The `MERGE_MIN_OVERLAP` property has been renamed to `MERGE_OVERLAP_THRESHOLD` so that setting it to a value of 0 will merge all regions that touch, regardless of how small the amount of overlap.
+- Refer to the [README](https://github.com/openmpf/openmpf-components/blob/master/python/EastTextDetection/README.md#properties) for details.
+
+<h2>MPFVideoCapture and MPFImageReader Tool Improvements</h2>
+
+- These tools now support a `ROTATION_FILL_COLOR` property for setting the fill color for pixels near the corners and edges of frames when performing non-orthogonal rotations. Previously, the color was hardcoded to `BLACK`. That is still the default setting for most components. Now the color can be set to `WHITE`, which is the default setting for the Tesseract component.
+- These tools now support a `ROTATION_THRESHOLD` property for adjusting the threshold at which the frame transformer performs rotation. Previously, the value was hardcoded to 0.1 degrees. That is still the default value. Rotation is not performed on any `ROTATION` value less than that threshold. The motivation is that some algorithms detect small rotations (for example, on structured text) when there is no rotation. In such cases rotating the frame results in fewer detections.
+- OpenMPF now uses FFmpeg when counting video frames. Refer to the Bug Fixes section below.
+
+<h2>Azure Cognitive Services (ACS) Form Detection Component</h2>
+
+- This new component utilizes the [Azure Cognitive Services Form Detection REST endpoint](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) to extract formatted text from documents (PDFs) and images. Refer to the [README](https://github.com/openmpf/openmpf-components/blob/master/python/AzureFormDetection/README.md) for details.
+- This component is capable of performing detections using a specified ACS endpoint URL. For example, different endpoints support receipt detection, business card detection, layout analysis, and support for custom models trained with or without labeled data.
+- This component may output the following detection properties depending on the endpoint, model, and media being processed: `TEXT`, `TABLE_CSV_OUTPUT`, `KEY_VALUE_PAIRS_JSON`, and `DOCUMENT_JSON_FIELDS`.
+
+<h2>Keyword Tagging Component</h2>
+
+- This new component performs the same keyword tagging behavior that was previously part of the Tesseract component, but does so on feed-forward tracks that generate detections with `TEXT` and `TRANSCRIPT` properties. Refer to the [README](https://github.com/openmpf/openmpf-components/blob/master/cpp/KeywordTagging/README.md) for details.
+- In addition to the Tesseract component, keyword tagging behavior has been removed from the Tika Text component and ACS OCR component.
+- Example pipelines have been added to the following components which make use of a final Keyword Tagging component stage:
+    - Tesseract
+    - Tika Text
+    - ACS OCR
+    - Sphinx
+    - ACS Speech
+
+<h2>Optionally Skip Media Inspection</h2>
+
+- The Workflow Manager will skip media inspection if all of the required media metadata is provided in the job request. The `MEDIA_HASH` and `MIME_TYPE` fields are always required. Depending on the media data type, other fields may be required or optional:
+    - Images
+        - Required: `FRAME_WIDTH`, `FRAME_HEIGHT`
+        - Optional: `HORIZONTAL_FLIP`, `ROTATION`
+    - Videos
+        - Required: `FRAME_WIDTH`, `FRAME_HEIGHT`, `FRAME_COUNT`, `FPS`, `DURATION`
+        - Optional: `HORIZONTAL_FLIP`, `ROTATION`
+    - Audio files
+        - Required: `DURATION`
 
-<h2 style="color:red">TensorRT Inference Server (TRTIS) Object Detection Component</h2>
+<h2>Updates</h2>
+
+- Update OpenMPF Python SDK exception handling for Python 3. Now instead of raising an `EnvironmentError`, which has been deprecated in Python 3, the SDK will raise an `mpf.DetectionError` or allow the underlying exception to be thrown.
+
+<h2>Bug Fixes</h2>
+
+- [[#1028](https://github.com/openmpf/openmpf/issues/1028)] OpenMPF can now properly handle Apple-optimized PNGs, which have a non-standard data chunk named CgBI before the IHDR chunk. The Workflow Manager uses [pngdefry](http://www.jongware.com/pngdefry.html) to convert the image into a standard PNG for processing. Before this fix, Tika would throw an error when trying to determine the MIME type of the Apple-optimized PNG.
+- [[#1130](https://github.com/openmpf/openmpf/issues/1130)] OpenMPF can now properly handle HEIC images. The Workflow Manager uses [libheif](https://github.com/strukturag/libheif) to convert the image into a standard PNG for processing. Before this fix, the HEIC image was sometimes falsely identified as a video and the Workflow Manager would fail to count the number of frames.
+- [[#1171](https://github.com/openmpf/openmpf/issues/1171)] The MIME type in the JSON output object is no longer null when there is a frame counting exception.
+- [[#1192](https://github.com/openmpf/openmpf/issues/1192)] When processing videos, the frame count is now obtained from both OpenCV and FFmpeg. The lower of the two is used. If they don't match, a `FRAME_COUNT` warning is generated. Before this fix, on some videos OpenCV would return frame counts that were magnitudes higher than the frames that could actually be read. This resulted in failing to process many video segments with a  `BAD_FRAME_SIZE` error.
+
+# OpenMPF 5.0.9: October 2020
+
+<h2>Bug Fixes</h2>
+
+- [[#1200](https://github.com/openmpf/openmpf/issues/1200)] The MPFVideoCapture and MPFImageReader tools now properly handle cropping to frame regions when the region coordinates fall outside of the frame boundary. There was a bug that would result in an OpenCV error. Note that the bug only occurred when cropping was not performed with rotation or flipping.
+
+# OpenMPF 5.0.8: October 2020
+
+<h2>Updates</h2>
+
+- The Tesseract component now supports a `TESSDATA_MODELS_SUBDIRECTORY` property. The component will look for tessdata files in `<MODELS_DIR_PATH>/<TESSDATA_MODELS_SUBDIRECTORY>`. This allows users to easily switch between `tessdata`, `tessdata_best`, and `tessdata_fast` subdirectories.
+
+<h2>Bug Fixes</h2>
+
+- [[#1199](https://github.com/openmpf/openmpf/issues/1199)] Added missing synchronized to InProgressBatchJobsService, which was resulting in some jobs staying `IN_PROGRESS` indefinitely.
+
+# OpenMPF 5.0.7: September 2020
+
+<h2>TensorRT Inference Server (TRTIS) Object Detection Component</h2>
+
+- This new component detects objects in images and videos by making use of an [NVIDIA TensorRT Inference Server](https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/) (TRTIS), and calculates features that can later be used by other systems to recognize the same object in other media. We provide support for running the server as a separate service during a Docker deployment, but an external server instance can be used instead.
+- By default, the ip_irv2_coco model is supported and will optionally classify detected objects using [COCO labels](https://github.com/openmpf/openmpf-components/blob/master/cpp/TrtisDetection/plugin-files/models/ip_irv2_coco/ip_irv2_coco.labels). Additionally, features can be generated for whole frames, automatically-detected object regions, and user-specified regions. Refer to the [README](https://github.com/openmpf/openmpf-components/blob/master/cpp/TrtisDetection/README.md).
+
+# OpenMPF 5.0.6: August 2020
+
+<h2>Enable OcvDnnDetection to Annotate Feed-forward Detections</h2>
+
+- The OcvDnnDetection component can now by configured to operate only on certain feed-forward detections and annotate them with supplementary information. For example, the following pipeline can be configured to generate detections that have both `CLASSIFICATION` and `COLOR` detection properties:
+
+```
+DarknetDetection (person + vehicle) --> OcvDnnDetection (vehicle color)
+```
+
+- For example:
+
+```
+  "detectionProperties": {
+    "CLASSIFICATION": "car",
+    "CLASSIFICATION CONFIDENCE LIST": "0.397336",
+    "CLASSIFICATION LIST": "car",
+    "COLOR": "blue",
+    "COLOR CONFIDENCE LIST": "0.93507; 0.055744",
+    "COLOR LIST": "blue; gray"
+  }
+```
+
+- The OcvDnnDetection component now supports the following properties:
+    - `CLASSIFICATION_TYPE`: Set this value to change the `CLASSIFICATION*` part of each output property name to something else. For example, setting it to `COLOR` will generate `COLOR`, `COLOR LIST`, and `COLOR CONFIDENCE LIST`. When handling feed-foward detections, the pre-existing `CLASSIFICATION*` properties will be carried over and the `COLOR*` properties will be added to the detection.
+    - `FEED_FORWARD_WHITELIST_FILE`: When `FEED_FORWARD_TYPE` is provided and not set to `NONE`, only feed-forward detections with class names contained in the specified file will be processed. For, example, a file with only "car" in it will result in performing the exclude behavior (below) for all feed-foward detections that do not have a `CLASSIFICATION` of "car".
+    - `FEED_FORWARD_EXCLUDE_BEHAVIOR`: Specifies what to do when excluding detections not specified in the `FEED_FORWARD_WHITELIST_FILE`. Acceptable values are:
+        - `PASS_THROUGH`: Return the excluded detections, without modification, along with any annotated detections.
+        - `DROP`: Don't return the excluded detections. Only return annotated detections.
+
+
+<h2>Updates</h2>
+
+- Make interop package work with Java 8 to better support exernal job producers and consumers.
+
+# OpenMPF 5.0.5: August 2020
+
+<h2>Updates</h2>
+
+- Configure Camel not to auto-acknowledge messages. Users can now see the number of pending messages in the ActiveMQ management console for queues consumed by the Workflow Manager.
+- Improve Tesseract OSD fallback behavior. This prevents selecting the OSD rotation from the fallback pass without the OSD script from the fallback pass.
+
+# OpenMPF 5.0.4: August 2020
+
+<h2>Updates</h2>
+
+- Retry job callbacks when they fail. The Workflow Manager now supports the `http.callback.timeout.ms` and `http.callback.retries` system properties.
+- Drop "duplicate paged in from cursor" DLQ messages.
+
+# OpenMPF 5.0.3: July 2020
+
+<h2>Updates</h2>
+
+- Update ActiveMQ to 5.16.0.
+
+# OpenMPF 5.0.2: July 2020
+
+<h2>Updates</h2>
 
-- <span style="color:red">TODO: This new component detects objects in images and videos by making use of an [NVIDIA TensorRT Inference Server](https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/) (TRTIS), and calculates features that can later be used by other systems to recognize the same object in other media. We provide support for running the server as a separate service during a Docker deployment, but an external server instance can be used instead. By default, the ip_irv2_coco model is supported and will optionally classify detected objects using [COCO labels](https://github.com/openmpf/openmpf-components/blob/master/cpp/trtisdetection/plugin-files/models/ip_irv2_coco.labels). Additionally, features can be generated for whole frames, automatically-detected object regions, and user-specified regions. Refer to the [README](https://github.com/openmpf/openmpf-components/blob/master/cpp/trtisdetection/README.md).</span>
--->
+- Disable video segmentation for ACS Speech Detection to prevent issues when generating speaker ids.
 
 # OpenMPF 5.0.1: July 2020
 
@@ -121,7 +258,7 @@
     - `[GET] /rest/actions`, `[GET] /rest/tasks`, `[GET] /rest/pipelines`
     - `[DELETE] /rest/actions`, `[DELETE] /rest/tasks`, `[DELETE] /rest/pipelines`
     - `[POST] /rest/actions` , `[POST] /rest/tasks`, `[POST] /rest/pipelines`
-- All of the endpoints above are new with the exception of `[GET] /rest/pipelines`. The endpoint has changed since the last version of OpenMPF. Some fields in the response JSON have been removed and renamed. Also, it now returns a collection of tasks for each pipelines. Refer to the REST API. 
+- All of the endpoints above are new with the exception of `[GET] /rest/pipelines`. The endpoint has changed since the last version of OpenMPF. Some fields in the response JSON have been removed and renamed. Also, it now returns a collection of tasks for each pipelines. Refer to the REST API.
 - `[GET] /rest/algorithms` can be used to get information about algorithms. Note that algorithms are tied to registered components, so to remove an algorithm you must unregister the associated component. To add an algorithm, start the associated component's Docker container so it self-registers with the Workflow Manager.
 
 <h2>Incomplete Actions, Tasks, and Pipelines</h2>
@@ -230,7 +367,7 @@
 
 <h2>Updates</h2>
 
-- Now silently discarding ActiveMQ DLQ "Suppressing duplicate delivery on connection" messages in addition to "duplicate from store" messages. 
+- Now silently discarding ActiveMQ DLQ "Suppressing duplicate delivery on connection" messages in addition to "duplicate from store" messages.
 
 # OpenMPF 4.1.5: March 2020
 
@@ -380,7 +517,7 @@ within a Docker container. This isolates the build and execution environment fro
 
 <h2>Late Additions: December 2019</h2>
 
-- Transitioned from using a mySQL persistent database to PostgreSQL to support users that use an external PostgreSQL database in the cloud. 
+- Transitioned from using a mySQL persistent database to PostgreSQL to support users that use an external PostgreSQL database in the cloud.
 - Updated the EAST component to support a `TEMPORARY_PADDING` and `FINAL_PADDING` property. The first property determines how much padding is added to detections during the non-maximum suppression or merging step. This padding is effectively removed from the final detections. The second property is used to control the final amount of padding on the output regions. Refer to the [README](https://github.com/openmpf/openmpf-components/blob/master/python/EastTextDetection/README.md#properties).
 
 # OpenMPF 4.0.0: February 2019

docs/docs/index.md

@@ -23,10 +23,13 @@ A list of algorithms currently integrated into the OpenMPF as distributed proces
 | Detection| Scene | OpenCV
 | Detection| Classification | OpenCV DNN
 | Detection/Tracking | Classification | Darknet
+| Detection/Tracking | Classification/Features | TensorRT
 | Detection| Text Region | EAST
 | Detection| Text (OCR) | Apache Tika
 | Detection| Text (OCR) | Tesseract OCR
 | Detection| Text (OCR) | Azure Cognitive Services
+| Detection| Form Structure (with OCR) | Azure Cognitive Services
+| Detection| Keywords | Boost Regular Expressions
 | Detection| Image (from document) | Apache Tika
 
 The OpenMPF exposes data processing and job management web services via a User Interface (UI). These services allow users to upload media, create media processing jobs, determine the status of jobs, and retrieve the artifacts associated with completed jobs. The web services give application developers flexibility to use the OpenMPF in their preferred environment and programming language.