Image Utils (#13)

* Initial commit

* initial commit.

* Add backbone for image_utils module. Also updated the setup.py file with the Pillow dependency.

* Added basic image pre processing utils.

* Added postprocessor code.

* added test image and code. modularized preprocessing.

* Changed variable names to avoid confusion.

* Fixed postprocessing code.

* Rename test_temp.py and add pytest, add verbosity support, and reformat

* Added functions for converting dtype and standardize/normalize.

* Switch to ABC structure, implement sanity checks, implement more transformations, and add some tests

* Update dimension checks, and add denormalization

* add documentation

* rearranged functions and other cosmetic changes.

* Fixed function signature in test script.

* Fixed issues from unit testing.

* fixed typo.

Co-Authored-By: kmh4321 <kmh4321@gmail.com>

* Updated test scripts.

* added individual test cases.

* Removed ABC.

* Removed DS_Store.

* Fixed pytest duplicate test calls.

* Fix travis CI flake issues.

* Fixes travis issues.

* Fixes travis issues.

* Fixes travis issues.

* WIP image utils rework

* WIP image utils rework

* added various tests, fixed some bugs, and implement standardize, normalize, rotate

* add flask error support

* add another layer to specifically customize error messages

* Fix flake8 issues.

* Fixed pytest error.

* Create a MAXImageProcessor and a regular ImageProcessor

* Propose move of MAXImageProcessor to `core/api.py`

* move to utils.py folder

* rewrite MAXImageProcessor and update tests

* [wip] updates nick

* Fix bug in standardize and normalize, enforce proper use and write corresponding tests

* add extra test for rotation

* add grayscale value tests and general image value bounds

* add more pixel value tests

* updates hong: add documentation for `apply transforms`, remove unused variable, and restructure tests`

* comments hong: further optimize tests and add documentation

* comments gitignore hong

* extend standardize functionality

* update standardize exceptions.

* Fix flake8 linting issue.

* feedback Nick

* Added specific flask exceptions.

* Flask exception handing.

* flake8.

* Added license headers.

* flask error messages.

* prohibit standardization of RGBA images

.gitignore

@@ -103,4 +103,10 @@ venv.bak/
 # mypy
 .mypy_cache/
 
-.idea/
+# local test files
+test_image_utils/
+.idea/
+
+# other
+*.DS_Store
+.vscode/

.travis.yml

@@ -15,7 +15,7 @@ install:
 before_script:
   - flake8 . --max-line-length=127
 script:
-  - true  # add other tests here
+  - python setup.py test  # add other tests here
 notifications:
   on_success: change
   on_failure: change  # `always` will be the setting once code changes slow down

maxfw/__init__.py

@@ -0,0 +1,15 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#

maxfw/core/__init__.py

@@ -1,2 +1,18 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
 from .app import MAXApp, MAX_API  # noqa
 from .api import *  # noqa
+from .utils import *  # noqa

maxfw/core/api.py

@@ -1,3 +1,18 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
 from .app import MAX_API
 from flask_restplus import Resource, fields
 

maxfw/core/app.py

@@ -1,3 +1,18 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
 import os
 from flask import Flask
 from flask_restplus import Api, Namespace

maxfw/core/default_config.py

@@ -1,3 +1,18 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
 # API metadata
 API_TITLE = 'Model Asset Exchange Microservice'
 API_DESC = 'An API for serving models'

maxfw/core/utils.py

@@ -0,0 +1,61 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+from flask import abort
+from maxfw.utils.image_utils import ImageProcessor
+
+
+def redirect_errors_to_flask(func):
+    """
+    This decorator function will capture all Pythonic errors and return them as flask errors.
+
+    If you are looking to disable this functionality, please remove this decorator from the `apply_transforms()` module
+    under the ImageProcessor class.
+    """
+
+    def inner(*args, **kwargs):
+        try:
+            # run the function
+            return func(*args, **kwargs)
+        except ValueError as ve:
+            if 'pic should be 2 or 3 dimensional' in str(ve):
+                abort(400, "Invalid input, please ensure the input is either "
+                      "a grayscale or a colour image.")
+        except TypeError as te:
+            if 'bytes or ndarray' in str(te):
+                abort(400, "Invalid input format, please make sure the input file format "
+                      " is a common image format such as JPG or PNG.")
+    return inner
+
+
+class MAXImageProcessor(ImageProcessor):
+    """Composes several transforms together.
+
+    Args:
+        transforms (list of ``Transform`` objects): list of transforms to compose.
+
+    Example:
+        >>> pipeline = ImageProcessor([
+        >>>     Rotate(150),
+        >>>     Resize([100,100])
+        >>> ])
+        >>> pipeline.apply_transforms(img)
+    """
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+    @redirect_errors_to_flask
+    def apply_transforms(self, img):
+        return super().apply_transforms(img)

maxfw/model/__init__.py

@@ -1 +1,16 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
 from .model import MAXModelWrapper  # noqa

maxfw/model/model.py

@@ -1,3 +1,19 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
 from abc import ABC, abstractmethod
 
 

maxfw/tests/__init__.py

@@ -0,0 +1,15 @@
+#
+# Copyright 2018-2019 IBM Corp. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#