Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V2.2.0? #766

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
12 changes: 12 additions & 0 deletions examples/idenprof/idenprof.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"0" : "chef",
"1" : "doctor",
"2" : "engineer",
"3" : "farmer",
"4" : "firefighter",
"5" : "judge",
"6" : "mechanic",
"7" : "pilot",
"8" : "police",
"9" : "waiter"
}
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,21 @@ A **DeepQuest AI** project [https://deepquestai.com](https://deepquestai.com)

ImageAI provides 4 different algorithms and model types to perform image prediction.
To perform image prediction on any picture, take the following simple steps. The 4 algorithms provided for
image prediction include **MobileNetV2**, **ResNet50**, **InceptionV3** and **DenseNet121**. Each of these
algorithms have individual model files which you must use depending on the choice of your algorithm. To download the
model file for your choice of algorithm, click on any of the links below:
image prediction include **MobileNetV2**, **ResNet50**, **InceptionV3** and **DenseNet121**. Each of these
algorithms have individual model files which you must use depending on the choice of your algorithm.

- **[MobileNetV2](https://github.com/OlafenwaMoses/ImageAI/releases/download/essentials-v5/mobilenet_v2.h5)** _(Size = 4.82 mb, fastest prediction time and moderate accuracy)_
- **[ResNet50](https://github.com/OlafenwaMoses/ImageAI/releases/download/essentials-v5/resnet50_imagenet_tf.2.0.h5)** by Microsoft Research _(Size = 98 mb, fast prediction time and high accuracy)_
- **[InceptionV3](https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/inception_v3_weights_tf_dim_ordering_tf_kernels.h5)** by Google Brain team _(Size = 91.6 mb, slow prediction time and higher accuracy)_
- **[DenseNet121](https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/DenseNet-BC-121-32.h5)** by Facebook AI Research _(Size = 31.6 mb, slower prediction time and highest accuracy)_

Great! Once you have downloaded this model file, start a new python project, and then copy the model file to your project
folder where your python files (.py files) will be . Download the image below, or take any image on your computer
and copy it to your python project's folder. Then create a python file and give it a name; an example is `FirstPrediction.py`.
Then write the code below into the python file:
- **[MobileNetV2]** _(Fastest prediction time and moderate accuracy)_
- **[ResNet50]** by Microsoft Research _(Fast prediction time and high accuracy)_
- **[InceptionV3]** by Google Brain team _(Slow prediction time and higher accuracy)_
- **[DenseNet121]** by Facebook AI Research _(Slower prediction time and highest accuracy)_

Great! Once you have chosen your model, start a new python project, and then copy the model file to your project
folder where your python files (.py files) will be. Download the image below, or take any image on your computer
and copy it to your python project's folder. Then create a python file and give it a name; an example is `FirstPrediction.py`.
Then write the code below into the python file:

### FirstPrediction.py
<div id="firstprediction" ></div>
<div id="firstprediction"></div>

```python
from imageai.Classification import ImageClassification
Expand All @@ -36,11 +35,10 @@ import os
execution_path = os.getcwd()

prediction = ImageClassification()
prediction.setModelTypeAsResNet50()
prediction.setModelPath(os.path.join(execution_path, "resnet50_imagenet_tf.2.0.h5"))
prediction.loadModel()
prediction.set_model_type('ResNet50')
prediction.load_model()

predictions, probabilities = prediction.classifyImage(os.path.join(execution_path, "1.jpg"), result_count=5 )
predictions, probabilities = prediction.classify_image(os.path.join(execution_path, "1.jpg"), result_count=5)
for eachPrediction, eachProbability in zip(predictions, probabilities):
print(eachPrediction , " : " , eachProbability)
```
Expand All @@ -49,11 +47,11 @@ Sample Result:
![](../../data-images/1.jpg)

```
convertible : 52.459555864334106
sports_car : 37.61284649372101
pickup : 3.1751200556755066
car_wheel : 1.817505806684494
minivan : 1.7487050965428352
convertible : 59.43046808242798
sports_car : 22.52226620912552
minivan : 5.942000821232796
pickup : 5.5309634655714035
car_wheel : 1.2574399821460247
```

The code above works as follows:
Expand All @@ -69,56 +67,42 @@ The above line obtains the path to the folder that contains your python file (in

```python
prediction = ImageClassification()
prediction.setModelTypeAsResNet50()
prediction.setModelPath(os.path.join(execution_path, "resnet50_imagenet_tf.2.0.h5"))
prediction.set_model_type('ResNet50')
```
In the lines above, we created and instance of the `ImagePrediction()` class in the first line, then we set the model type of the prediction object to ResNet by caling the `.setModelTypeAsResNet50()` in the second line and then we set the model path of the prediction object to the path of the model file (`resnet50_imagenet_tf.2.0.h5`) we copied to the python file folder in the third line.
In the lines above, we created and instance of the `ImagePrediction()` class in the first line, then set the model type of the prediction object to ResNet by caling the `.set_model_type('ResNet50')` in the second line.

```python
predictions, probabilities = prediction.classifyImage(os.path.join(execution_path, "1.jpg"), result_count=5 )
predictions, probabilities = prediction.classify_image(os.path.join(execution_path, "1.jpg"), result_count=5 )
```

In the above line, we defined 2 variables to be equal to the function called to predict an image, which is the `.classifyImage()` function, into which we parsed the path to our image and also state the number of prediction results we want to have (values from 1 to 1000) parsing `result_count=5`. The `.classifyImage()` function will return 2 array objects with the first (**predictions**) being an array of predictions and the second (**percentage_probabilities**) being an array of the corresponding percentage probability for each prediction.
In the above line, we defined 2 variables to be equal to the function called to predict an image, which is the `.classify_image()` function, into which we parsed the path to our image and also state the number of prediction results we want to have (values from 1 to 1000) parsing `result_count=5`. The `.classify_image()` function will return 2 array objects with the first (**predictions**) being an array of predictions and the second (**probabilities**) being an array of the corresponding percentage probability for each prediction.

```python
for eachPrediction, eachProbability in zip(predictions, probabilities):
print(eachPrediction, " : " , eachProbability)
print(eachPrediction, ':' , eachProbability)
```
The above line obtains each object in the **predictions** array, and also obtains the corresponding percentage probability from the **percentage_probabilities**, and finally prints the result of both to console.
The above line obtains each object in the **predictions** array, and also obtains the corresponding percentage probability from the **probabilities**, and finally prints the result of both to console.


### Prediction Speed
<!-- ### Prediction Speed
<div id="predictionspeed"></div>

**ImageAI** now provides prediction speeds for all image prediction tasks. The prediction speeds allow you to reduce the time of prediction at a rate between 20% - 60%, and yet having just slight changes but accurate prediction results. The available prediction speeds are **"normal"**(default), **"fast"**, **"faster"** and **"fastest"**.
All you need to do is to state the speed mode you desire when loading the model as seen below.

```python
prediction.loadModel(prediction_speed="fast")
prediction.load_model(prediction_speed="fast")
```

To observe the differences in the prediction speeds, look below for each speed applied to multiple prediction with time taken to predict and predictions given. The results below are obtained from predictions performed on a Windows 8 laptop with Intel Celeron N2820 CPU, with processor speed of 2.13GHz
To observe the differences in the prediction speeds, look below for each speed applied to multiple prediction with time taken to predict and predictions given. Note: Speeds will vary from computer to computer

**Prediction Speed = "normal" , Prediction Time = 5.9 seconds**
**Prediction Speed = "normal" , Prediction Time = 2 seconds**
```
convertible : 52.459555864334106
sports_car : 37.61284649372101
pickup : 3.1751200556755066
car_wheel : 1.817505806684494
minivan : 1.7487050965428352
-----------------------
toilet_tissue : 13.99008333683014
jeep : 6.842949986457825
car_wheel : 6.71963095664978
seat_belt : 6.704962253570557
minivan : 5.861184373497963
-----------------------
bustard : 52.03368067741394
vulture : 20.936034619808197
crane : 10.620515048503876
kite : 10.20539253950119
white_stork : 1.6472270712256432
-----------------------
convertible : 59.43046808242798
sports_car : 22.52226620912552
minivan : 5.941995605826378
pickup : 5.5309634655714035
car_wheel : 1.2574387714266777
```

**Prediction Speed = "fast" , Prediction Time = 3.4 seconds**
Expand Down Expand Up @@ -187,7 +171,7 @@ great_grey_owl : 0.0699841941241175
-----------------------
```

**PLEASE NOTE:** When adjusting speed modes, it is best to use models that have higher accuracies like the DenseNet or InceptionV3 models, or use it in case scenarios where the images predicted are iconic.
**PLEASE NOTE:** When adjusting speed modes, it is best to use models that have higher accuracies like the DenseNet or InceptionV3 models, or use it in case scenarios where the images predicted are iconic. -->


### Image Input Types
Expand All @@ -196,25 +180,24 @@ great_grey_owl : 0.0699841941241175
Previous version of **ImageAI** supported only file inputs and accepts file paths to an image for image prediction.
Now, **ImageAI** supports 3 input types which are **file path to image file**(default), **numpy array of image** and **image file stream**.
This means you can now perform image prediction in production applications such as on a web server and system
that returns file in any of the above stated formats.
that returns file in any of the above stated formats.

To perform image prediction with numpy array or file stream input, you just need to state the input type
in the `.classifyImage()` function. See example below.
in the `.classify_image()` function. See example below.

```python
predictions, probabilities = prediction.classifyImage(image_array, result_count=5 , input_type="array" ) # For numpy array input type
predictions, probabilities = prediction.classifyImage(image_stream, result_count=5 , input_type="stream" ) # For file stream input type
predictions, probabilities = prediction.classify_image(image_array, result_count=5 , input_type="array" ) # For numpy array input type
predictions, probabilities = prediction.classify_image(image_stream, result_count=5 , input_type="stream" ) # For file stream input type
```

### Prediction in MultiThreading
<div id="threadprediction"></div>

When developing programs that run heavy task on the deafult thread like User Interfaces (UI),
you should consider running your predictions in a new thread. When running image prediction using ImageAI in
a new thread, you must take note the following:
- You can create your prediction object, set its model type, set model path and json path
outside the new thread.
- The `.loadModel()` must be in the new thread and image prediction (`classifyImage()`) must take place in th new thread.
you should consider running your predictions in a new thread. When running image prediction using ImageAI in
a new thread, you must take note the following:
- You can create your prediction object, set its model type, set model path and json path outside the new thread.
- The `.load_model()` must be in the new thread and image prediction (`classify_image()`) must take place in th new thread.

Take a look of a sample code below on image prediction using multithreading:
```python
Expand All @@ -225,8 +208,7 @@ import threading
execution_path = os.getcwd()

prediction = ImageClassification()
prediction.setModelTypeAsResNet()
prediction.setModelPath( os.path.join(execution_path, "resnet50_imagenet_tf.2.0.h5"))
prediction.set_model_type('ResNet50')

picturesfolder = os.environ["USERPROFILE"] + "\\Pictures\\"
allfiles = os.listdir(picturesfolder)
Expand All @@ -235,24 +217,24 @@ class PredictionThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
prediction.loadModel()
prediction.load_model()
for eachPicture in allfiles:
if eachPicture.endswith(".png") or eachPicture.endswith(".jpg"):
predictions, percentage_probabilities = prediction.predictImage(picturesfolder + eachPicture, result_count=1)
predictions, percentage_probabilities = prediction.classify_image(picturesfolder + eachPicture, result_count=1)
for prediction, percentage_probability in zip(predictions, probabilities):
print(prediction , " : " , percentage_probability)

predictionThread = PredictionThread ()
predictionThread = PredictionThread()
predictionThread.start()

```


### Documentation
<!-- ### Documentation

We have provided full documentation for all **ImageAI** classes and functions in 3 major languages. Find links below:**

* Documentation - **English Version [https://imageai.readthedocs.io](https://imageai.readthedocs.io)**
* Documentation - **Chinese Version [https://imageai-cn.readthedocs.io](https://imageai-cn.readthedocs.io)**
* Documentation - **French Version [https://imageai-fr.readthedocs.io](https://imageai-fr.readthedocs.io)**
* Documentation - **French Version [https://imageai-fr.readthedocs.io](https://imageai-fr.readthedocs.io)** -->

44 changes: 44 additions & 0 deletions imageai/Classification/Custom/Changes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
### Changes in ImageAI V2.2.0
<div id="customchanges"></div>

ImageAI hasn't been updated since TensorFlow 2.4.0 and were on 2.9.1 now! It's definitely time for some updates and in ImageAI 2.2.0 there were some big changes that involve making using ImageAI as seamless as possible for new users who just want something that works without much configuration while still allowing configuration for users who want to dive deeper into image classification but still aren't ready to build their own models from the ground up or just want something that's easier to use and configure without having to completely re-write your program (I'm the latter and that's why I decided to rewrite it for myself and then wanted to share with everyone else)

NOTE: The custom image classification has been updated quite a bit with changes mostly happening just to get everything working with current versions of tensorflow. With that said, tests were not re-run with the newest version of tensor flow so some of the specific timings of things most likely will not be accurate anymore (they'll usually be faster) but keep that in mind throughout this readme.

**Now time for the changes!**

ImageAI Custom Classification Version 2.2.0 Changes:

Setting models:
- The four model types included with ImageAI are currently still the same as in v2.1.6 however to streamline the package code, make the end user process more simple, and future proof ImageAI, models will no longer be set with setModelTypeAsResNet50(), etc. Models will now be set using set_model_type() with the model you'd like to use ex: trainer.setModelTypeAs('ResNet50')
- On top of changing how the model type is set, setting a model type is no longer required. In almost every case that I've seen, people tend to use ResNet50 so that is now the default model that will be set if set_model_type() is not called. Note: This is mostly to make the experience easier for beginner users

Model training:
- Removed 'num_objects' from trainModel(), this will get calculate based on how many class folders you have in your dataset directory/directories
- Removed 'transfer_with_full_learning' from trainModel() as it wasn't being used
- 'continue_from_model' now loads a full model and trains from that
- Changed 'enhance_data' to 'preprocess_layers' in trainModel() to be more fitting to the process happening
- Added 'show_training_graph' to trainModel() which is set to False by default. When set to True a graph plotting accuracy with validation accuracy as well as loss with validation loss will show at the end of training
- Moved preprocessing to before the selected model as this is what is recommended in official tensorflow and keras documentation and should improve accuracy when training
- Added RandomZoom() and RandomRotation() to preprocessing to further eliminate overfitting during training
- Removed ImageDataGenerator() as this is depreciated, proprocessing will take the place of this along with rescaling before each model
- Rescaling has been used instead of each models build in preprocessing function as there were issues with training and prediction when using them
- Removed flow_from_directory() as this has been depreceated and image_dataset_from_directory will take its place (this also has the functionality of automatically splitting a single dataset into training and validation datasets automatically)
- Added caching and prefetching of datasets to speed up the training process
- Setting up the chosen model has been simplified and updated to match current best practices in TensorFlow
- On all models 'include_top' has been set to false because of errors when it was set to True
- Saved weights now also contain the name of the model used during training as well as validation accuracy
- Removed tensorboard callback as it wasn't being used

Custom Image Classification:
- set_json_path() is no longer required as long as the json file for the corresponding model is located in the 'json' folder and is named 'model_class.json'
- Added get_json() which will check if json path is set, if not set will check for 'json/model_class.json' and raise and error if neither are found

Loading Model:
- Simplified code when setting up the model for prediction

Classify Image:
- Changed all keras.preprocessing.image to keras.utils as keras.preprocessing is deprecated
- Added extra processing of the prediction with tf.nn.softmax() as raw data was unreadable
- Unfortunantly due to updates in Tensorflow models from previous generations of ImageAI are no longer going to be supported as the specific model used during training needs to be used during prediction. If you have a Tensorflow SavedModel or HDF5 full model, it can now be used for prediction regardless of if the model was trained in ImageAI or not. If you do not have the 'save_full_model' set to true on models you'd like to continue using, run a few epochs of transfer learning with that model and the 'save_full_model' set to True and it can be carried over to ImageAI V2.2.0
(This is still being looked into to see if there is a way to transfer models over)
Loading