Convert to modular application. Upgrade tensorflow library. Reduce ap…

…plication bundle size.
kinhong · Mar 15, 2024 · d570555 · d570555
1 parent 401a087
commit d570555
Show file tree

Hide file tree

Showing 10 changed files with 120 additions and 89 deletions.
diff --git a/.idea/misc.xml b/.idea/misc.xml
diff --git a/.idea/vcs.xml b/.idea/vcs.xml
diff --git a/README.md b/README.md
@@ -4,23 +4,23 @@
 
 ## Introduction
 
-**OpenLabeler** is an open source application for annotating objects. It can generate the PASCAL VOC format XML annotation file for artificial intelligence and deep learning training. A unique aspect of this application is its ability to use inference (with [TensorFlow](https://www.tensorflow.org)) to help improve accuracy and speed up the annotation process.
+**OpenLabeler** is an open-source application for annotating objects. It can generate the PASCAL VOC format XML annotation file for artificial intelligence and deep learning training. This application's unique aspect is its ability to use inference (with [TensorFlow](https://www.tensorflow.org)) to improve accuracy and speed up the annotation process.
 
-OpenLabeler is written in [OpenJDK](https://openjdk.java.net)/[OpenJFX](https://openjfx.io) (version 18.x).
+OpenLabeler is written in [OpenJDK](https://openjdk.java.net)/[OpenJFX](https://openjfx.io) (version 21.x).
 
 ![Application](assets/app.png)
 
 A few highlights:
 
-* Fast Labeling (no need for Open/Save File actions)
+* Fast labeling (no need for Open/Save File actions)
 ![General Preferences](assets/pref-general.png)
 * Multi-level undo/redo
 * Annotation "hints" (using TensorFlow inference) 
-* Pre-built installation packages for macOS (tested on macOS Mojave), Linux (tested on Ubuntu 18.04 LTS), and Windows (tested on Windows 10 Pro)
+* Pre-built installation packages for macOS (tested on macOS Sonoma), Linux (tested on Ubuntu 20.04 LTS), and Windows (tested on Windows 10 Pro)
 
 ## Inference
 
-OpenLabeler can help improve the speed and accuracy of annotation by providing labeling "hints" from a saved model using TensorFlow.
+OpenLabeler can help improve the speed and accuracy of annotation by offering labeling "hints" from a saved model using TensorFlow (currently, only x86/x86_64 machines are supported).
 
 For example, you have thousands of images to annotate. After labeling the first 300 or so images, you could train a model using these 300 samples, then configure OpenLabeler to use this intermediary model to give you labeling suggestions for the remaining images, thereby speeding up the annotation task.
 
@@ -32,20 +32,20 @@ The **Saved Model Location** is the *folder* where the `.pb` file is located. If
 
 OpenLabeler supports graphs with the `image_tensor` and `encoded_image_string_tensor` operations/input types.
 
+The protobuf sources is located in https://github.com/tensorflow/models/tree/master/research/object_detection/protos
+
 ## Training Support
 
 *Note: This is currently an experimental feature.*
 
 OpenLabeler can be used to start/stop a training process in TensorFlow running inside a [Docker](https://www.docker.com) container. Containers with [TensorFlow 2](https://www.tensorflow.org/install/docker) and [Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) dependencies have been pre-built for your convenience. To use this feature:
 
 1. [Install Docker](https://docs.docker.com/install) on your host machine
-2. *Windows only* Open Docker > Settings > General and check "Expose daemon on tcp://localhost:2375 without TLS"
-3. *Windows only* Create a `<home>\.docker-java.properties` file. Add a `DOCKER_HOST=tcp://localhost:2375 entry`. See [reference](https://github.com/docker-java/docker-java)
-4. Choose a pre-built, `kinhong/openlabeler:tf-2.3.1` or `kinhong/openlabeler:tf-2.3.1-gpu`, [docker image](https://cloud.docker.com/repository/docker/kinhong/openlabeler/tags) from [Docker Hub](https://hub.docker.com/) and pull it to your docker host
-5. Download a base model from the [TensorFlow 2 Detection Model Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md) for transfer learning
-6. Configure the Training Preference settings (and add the label map entries)
+2. Choose a pre-built, `kinhong/openlabeler:tf-2.3.1` or `kinhong/openlabeler:tf-2.3.1-gpu`, [docker image](https://cloud.docker.com/repository/docker/kinhong/openlabeler/tags) from [Docker Hub](https://hub.docker.com/) and pull it to your docker host
+3. Download a base model from the [TensorFlow 2 Detection Model Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md) for transfer learning
+4. Configure the Training Preference settings (and add the label map entries)
 ![Train Preferences](assets/pref-train.png)
-7. You can then start, stop, continue, restart training, or export the inference graph
+5. You can then start, stop, continue, restart training, or export the inference graph
 
 ## Shortcut Keys
 
@@ -105,7 +105,7 @@ First make sure the environment variable JAVA_HOME has been set accordingly
 
 ### macOS
 
-1. Download and install [OpenJDK 18](http://jdk.java.net/18)
+1. Download and install [OpenJDK 21](http://jdk.java.net/21)
 2. Download and install [Maven](https://maven.apache.org/install.html)
 ```
 cd <openlabeler>
@@ -116,12 +116,12 @@ The macOS .pkg installer can be found under the app/target/package directory.
 ### Linux
 ```
 sudo add-apt-repository ppa:openjdk-r/ppa \
-sudo apt-get update -q \
-sudo apt install -y openjdk-18-jdk
-
-sudo apt-get install maven
+sudo apt update -q \
+sudo apt install -y openjdk-21-jdk
 
-sudo apt-get install fakeroot
+sudo apt install maven
+sudo apt install binutils
+sudo apt install fakeroot
 
 cd <openlabeler>
 mvn clean package -Drevision=x.y.z
@@ -130,8 +130,10 @@ The Linux .deb bundle can be found under the app/target/package directory.
 
 ### Windows
 
-1. Download [OpenJDK 18](http://jdk.java.net/18/) for Windows and unzip to a directory with no spaces (e.g., `C:\java\jdk-18`)
+1. Download [OpenJDK 21](https://learn.microsoft.com/en-us/java/openjdk/download#openjdk-21) for Windows and unzip to a directory with no spaces (e.g., `C:\java\jdk-21`)
 2. Download [Maven](https://maven.apache.org/download.cgi) and unzip to a directory with no spaces (e.g., `C:\java\apache-maven`)
+3. Download [Wix Toolset](https://github.com/wixtoolset/wix3) and unzip to a directory with no spaces (e.g., `c:\wix`)
+3. Make sure java, mvn and wix executables are in your Windows PATH (e.g., `set PATH=%PATH%;C:\java\jdk-21\bin;C:\java\apache-maven\bin;c:\wix`)
 
 ```DOS .bat
 cd <openlabeler>

diff --git a/app/pom.xml b/app/pom.xml
@@ -56,19 +56,13 @@
         </dependency>
         <dependency>
             <groupId>org.tensorflow</groupId>
-            <artifactId>tensorflow-core-platform</artifactId>
-            <version>0.4.1</version>
+            <artifactId>tensorflow-core-api</artifactId>
+            <version>0.5.0</version>
         </dependency>
         <dependency>
             <groupId>org.tensorflow</groupId>
             <artifactId>proto</artifactId>
             <version>1.15.0</version>
-            <exclusions>
-                <exclusion>
-                    <artifactId>protobuf-java</artifactId>
-                    <groupId>com.google.protobuf</groupId>
-                </exclusion>
-            </exclusions>
         </dependency>
         <dependency>
             <groupId>com.google.protobuf</groupId>
@@ -107,7 +101,7 @@
         </dependency>
         <dependency>
             <groupId>com.github.docker-java</groupId>
-            <artifactId>docker-java</artifactId>
+            <artifactId>docker-java-core</artifactId>
             <version>3.3.6</version>
         </dependency>
         <dependency>
@@ -198,6 +192,7 @@
                 <artifactId>maven-resources-plugin</artifactId>
                 <version>3.3.1</version>
                 <configuration>
+                    <propertiesEncoding>UTF-8</propertiesEncoding>
                     <nonFilteredFileExtensions>
                         <nonFilteredFileExtension>png</nonFilteredFileExtension>
                         <nonFilteredFileExtension>icns</nonFilteredFileExtension>
@@ -206,6 +201,7 @@
                 </configuration>
             </plugin>
             <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-dependency-plugin</artifactId>
                 <version>3.6.1</version>
                 <executions>
@@ -218,29 +214,13 @@
                         <configuration>
                             <outputDirectory>${project.build.directory}/libs</outputDirectory>
                             <includeScope>runtime</includeScope>
-                            <excludeGroupIds>org.openjfx</excludeGroupIds>
-                        </configuration>
-                    </execution>
-                    <execution>
-                        <id>copy-modules</id>
-                        <phase>prepare-package</phase>
-                        <goals>
-                            <goal>copy-dependencies</goal>
-                        </goals>
-                        <configuration>
-                            <outputDirectory>${project.build.directory}/mods</outputDirectory>
-                            <includeScope>runtime</includeScope>
-                            <includeGroupIds>org.openjfx</includeGroupIds>
                         </configuration>
                     </execution>
                 </executions>
             </plugin>
         </plugins>
     </build>
     <properties>
-        <project.build.add-modules>
-            java.base,java.logging,java.desktop,java.prefs,java.xml,javafx.base,javafx.graphics,javafx.controls,javafx.fxml,javafx.web,javafx.media,javafx.swing
-        </project.build.add-modules>
         <project.vendor>OpenLabeler</project.vendor>
     </properties>
     <profiles>
@@ -251,6 +231,14 @@
                     <family>mac</family>
                 </os>
             </activation>
+            <dependencies>
+                <dependency>
+                    <groupId>org.tensorflow</groupId>
+                    <artifactId>tensorflow-core-api</artifactId>
+                    <version>0.5.0</version>
+                    <classifier>macosx-x86_64</classifier>
+                </dependency>
+            </dependencies>
             <build>
                 <plugins>
                     <plugin>
@@ -304,8 +292,6 @@
                                         <!-- jlink arguments -->
                                         <argument>--module-path</argument>
                                         <argument>${java.home}/jmods:${project.build.directory}/mods</argument>
-                                        <argument>--add-modules</argument>
-                                        <argument>${project.build.add-modules}</argument>
                                     </arguments>
                                 </configuration>
                             </execution>
@@ -325,6 +311,14 @@
                     <name>Linux</name>
                 </os>
             </activation>
+            <dependencies>
+                <dependency>
+                    <groupId>org.tensorflow</groupId>
+                    <artifactId>tensorflow-core-api</artifactId>
+                    <version>0.5.0</version>
+                    <classifier>linux-x86_64</classifier>
+                </dependency>
+            </dependencies>
             <build>
                 <plugins>
                     <plugin>
@@ -376,8 +370,6 @@
                                         <!-- jlink arguments -->
                                         <argument>--module-path</argument>
                                         <argument>${java.home}/jmods:${project.build.directory}/mods</argument>
-                                        <argument>--add-modules</argument>
-                                        <argument>${project.build.add-modules}</argument>
                                     </arguments>
                                 </configuration>
                             </execution>
@@ -396,6 +388,14 @@
                     <family>windows</family>
                 </os>
             </activation>
+            <dependencies>
+                <dependency>
+                    <groupId>org.tensorflow</groupId>
+                    <artifactId>tensorflow-core-api</artifactId>
+                    <version>0.5.0</version>
+                    <classifier>windows-x86_64</classifier>
+                </dependency>
+            </dependencies>
             <build>
                 <plugins>
                     <plugin>
@@ -448,8 +448,6 @@
                                         <!-- jlink arguments -->
                                         <argument>--module-path</argument>
                                         <argument>${java.home}/jmods;${project.build.directory}/mods</argument>
-                                        <argument>--add-modules</argument>
-                                        <argument>${project.build.add-modules}</argument>
                                     </arguments>
                                 </configuration>
                             </execution>

diff --git a/app/src/main/java/com/easymobo/openlabeler/tensorflow/ObjectDetector.java b/app/src/main/java/com/easymobo/openlabeler/tensorflow/ObjectDetector.java
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2022. Kin-Hong Wong. All Rights Reserved.
+ * Copyright (c) 2024. Kin-Hong Wong. All Rights Reserved.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -26,15 +26,16 @@
 import org.apache.commons.io.FileUtils;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang3.ArrayUtils;
+import org.tensorflow.Result;
 import org.tensorflow.SavedModelBundle;
 import org.tensorflow.Tensor;
 import org.tensorflow.TensorFlow;
-import org.tensorflow.framework.DataType;
-import org.tensorflow.framework.MetaGraphDef;
-import org.tensorflow.framework.SignatureDef;
-import org.tensorflow.framework.TensorInfo;
 import org.tensorflow.ndarray.Shape;
 import org.tensorflow.ndarray.buffer.DataBuffers;
+import org.tensorflow.proto.framework.DataType;
+import org.tensorflow.proto.framework.MetaGraphDef;
+import org.tensorflow.proto.framework.SignatureDef;
+import org.tensorflow.proto.framework.TensorInfo;
 import org.tensorflow.types.TFloat32;
 import org.tensorflow.types.TString;
 import org.tensorflow.types.TUint8;
@@ -143,7 +144,7 @@ else if (!savedModelFile.exists() && path == null) {
                 LOG.info(savedModelFile.toString() + " does not exist");
             }
         }
-        catch (Exception ex) {
+        catch (Throwable ex) {
             LOG.log(Level.SEVERE, "Unable to update " + path, ex);
         }
         return null;
@@ -156,7 +157,7 @@ public List<HintModel> detect(File imageFile) throws IOException {
                 return Collections.emptyList();
             }
             List<HintModel> hints = new ArrayList();
-            List<Tensor> outputs;
+            Result result;
             Tensor input = null;
             String operation = "";
             BufferedImage img = ImageIO.read(imageFile);
@@ -178,7 +179,7 @@ else if (sig.containsInputs("input_tensor")) {
                     operation = tensorInfo.getName();
                     input = tensorInfo.getDtype() == DataType.DT_STRING ? makeImageStringTensor(imageFile) : makeImageTensor(img);
                 }
-                outputs = model.session()
+                result = model.session()
                       .runner()
                       .feed(operation, input)
                       .fetch(sig.getOutputsOrThrow("detection_scores").getName())
@@ -191,9 +192,9 @@ else if (sig.containsInputs("input_tensor")) {
                     input.close();
                 }
             }
-            try (TFloat32 scoresT = (TFloat32)outputs.get(0);
-                 TFloat32 classesT = (TFloat32)outputs.get(1);
-                 TFloat32 boxesT = (TFloat32)outputs.get(2)) {
+            try (TFloat32 scoresT = (TFloat32)result.get(0);
+                 TFloat32 classesT = (TFloat32)result.get(1);
+                 TFloat32 boxesT = (TFloat32)result.get(2)) {
                 // All these tensors have:
                 // - 1 as the first dimension
                 // - maxObjects as the second dimension

diff --git a/app/src/main/java/com/easymobo/openlabeler/tensorflow/TFRecordCreator.java b/app/src/main/java/com/easymobo/openlabeler/tensorflow/TFRecordCreator.java
@@ -24,8 +24,8 @@
 import jakarta.xml.bind.Unmarshaller;
 import org.apache.commons.codec.binary.Hex;
 import org.apache.commons.io.IOUtils;
-import org.tensorflow.example.*;
 import org.tensorflow.hadoop.util.TFRecordWriter;
+import org.tensorflow.proto.example.*;
 
 import java.io.DataOutputStream;
 import java.io.File;