Plant Leaf Diseases Diagnosis with Deep Learning

Plant disease identification relies on the experience and expertise of farmers or gardeners. As a home gardener or houseplant hobbyist, most of the time the required experience and expertise is missing. A machine learning based plant leaf diseases identification tool becomes very helpful.

I have a firsthand experience where I helped my mother treat her ailing indoor plants. Unaware of the exact disease, my mother used several pesticides, which made everyone flee the house. It wasn't until we got advice from a plant expert, who suggested an appropriate treatment, that the flower began to recuperate. How I wish there was a plant disease identification tool.

1. Data

In order to build a machine learning model for plant disease diagnosis, images of plant leaves with different diseases are required. After some search online, I found the following dataset on Mendeley Data which was originally from a published journal paper.

• Image Data

• Journal Paper

2. Method

The problem at hand is to build a supervised classification model. This is a fundamental task in computer vision and several models have been developed for this purpose:

1. Convolutional Neural Networks: CNNs are the backbone of most modern image classification techniques. They can automatically learn the spatial features from images.

2. CNN-based Models: ResNet (Residual Networks) developed by Microsoft, GoogLeNet introduced by Google, VGGNet developed by Visual Graphics Group at Oxford and etc.

3. Transformer-based Models for Vision: Originally designed for natural language processing, transformers have been adapted to vision tasks with promising results.

In this work, a 9-layer CNN model (structure below) was selected for the plant leaf disease diagnosis task. CNN models are the foundation of more advanced models and they are well developed and widely used in the deep learning community.

3. Data Challenges

After wrangling the raw image data, there are 39 classes of images with very imbalanced number of samples in each class.

Here are the numbers of images in each class for the training data and testing data.

By carefully selecting model metrics, the imbalanced testing data may not be a big issue. However, the imbalanced training data centainly will have negative impacts on model building.

Besides the imbalanced class issue, there are several classes with a very limited number of images, the smallest number only 122. With such limited data, it is hard for any model to give accurate predictions.

To address the limiting sample issue and imbalanced class issue in the training data:

• Solution 1: The ideal approach will be collecting more data for the classes with limited amounts of samples. In this work, inspired by the original journal paper, data augmentation using existing data was implemented to increase number of samples.

• Solution 2: With unbalanced data, ituitively we can down-scale or up-scale the data. The widely used solution in practice is Boostrapping, which mitigates the data unbalance issue and improves model accuracy. Therefore, boostrapping was tried in this work.

Data Augmentation Notebook

Boostrapping Notebook

4. Simple Pre-processing

Pre-processing on image data was kept simple for this work. Try to decrease the computational cost but maintain the key features in the original data as much as possible.

Here is the step-by-step pre-processing on the input data:

Note that color images were used for modeling in this work. Gray scale is evaluated but not tried for modeling yet.

Pre-processing Notebook

5. Modeling and Evaluation

The CNN model was built using TensorFlow.Keras.Models. Model structure and parameters were kept simple and some from the journal paper.

The focus was to test performance of models built from the following approaches:

Accuracy, cross_entropy, classification report and confusion matrix were selected as the metrics for model evaluation.

Note that for model_3 and model_4, a majority vote was used to generate the final output from the 5 different Bootstrapping models.

WINNER: model_1 & model_4

model_1 had the best image quality since there was no data augmentation. model_4 had the best balanced classes with adquate samples. Therefore, both data quality and quantity play an important role on how good the model is.

Modeling Notebook

6. Hyper-parameter Tuning

The original CNN model was tuned using the hyperband optimization algorithm to improve model performance with approach 4. The hyperband optimization better utilized the resources in a more efficient way compared to grid search or random search. kears_tuner.Hyperband packadge was used to implement the hyperband optimization. The following parameters were tested:

• filter numbers in the 2nd Conv2d
• filter numbers in the 3rd Conv2d
• rate in Dropout layer
• optimizer method in model.compile

Here is the optimized CNN structure.

The tuned model provided better predictions on test data. Compared to the original setup, more filter numbers in the last convolution layer and less dropout were prefered by the hyperband optimization algorithm.

Hyper-parameter tuning notebook

7. Future Improvements

There is so much more to improve for this task.

Modeling:

Model hyper parameter tuning, such as more filters in convolution layers, more units in the fully connected layer, learning rate in optimizer, epoch numbers and etc.

Feature Engineering:

• Bootstrapping to draw more images from data without augmentation, such as to draw 500 images for each class.
• If improvement is observed for the step above, try drawing more images for each class to find the sweet spot.
• Bootstrapping to draw more images from data with augmentation, say 1500 for each class to see if there are any benefits.
• Find a balance between bootstrapping sample size the ensembling times, small sample size requiring more ensembling times.

Pre-processing:

After settling down on the model and feature engineering part, try using gray scale images as the input to reduce computational cost.

Data Augmentation:

• Multiple data augmentation methods were used in this work, such as rotation, flip, scaling, adding noises, etc. Try to limit to one augmentation method to see if certain augmentation improves model performance.
• Explore other augmentation techniques.

Input Data:

Search online to collect more images for the classes with a limited number of samples.

8. Acknowledgement

Thanks to Nicolas Renotte for his fun and engaging YouTube Video on Build a Deep CNN Image Classifier and Raghunandan Patthar for being a super supporting Springboard mentor.