Plant Disease Detection Using Advanced Convolutional Neural Networks with Region of Interest Awareness

Introduction

Agriculture is very crucial domain across the globe as it takes care of human survival process from time immemorial. In fact, in India, it is very crucial domain as majority of population depend on ag- riculture or its related industries. The advent of deep learning has paved way for more efficient computer vision applications. In ag- ricultural crop monitoring with technology driven approaches, it is indispensable to have plant disease identification. Recent research reveals that Convolutional Neural Network (CNN) is most suitable deep learning method to process leaf images for detecting diseas- es. As symptoms of leaf disease appear in specific area, consider- ing entire leaf for processing incurs more computational cost and time besides deteriorating performance due to inadequate quality of training.Deep learning models have been around for improv- ing image processing based leaf disease detection. As explored in many CNN based models in it is observed that CNN models are most appropriate for disease detection in agricultural crops [1, 5, 8, 10].

Proposed a Region of Interest (ROI) based approach along with ResNet-50 CNN for identification of plant leaf diseases. Proposed an improved faster RCNN model for object detection towards to- mato disease detection [35, 39]. They also make use of ROI to- wards improving performance. From the literature, it is understood that there are different approaches in deep learning models and pre-trained CNN models found. However, there is little research on ROI based approach with CNN. It is the motivation behind the work in this paper. Our contributions in this paper are as follows.

1. Proposed a methodology for plant disease detection using ad- vanced convolutional neural networks with region of interest awareness.

2. An algorithm named ROI Feature Map Creation (ROI-FMC) is proposed and implemented to create ROI feature map that plays crucial role in leaf disease detection with ROI awareness.

3. Another algorithm known as ROI based Deep CNN with Trans- fer Learning for Leaf Disease Prediction (ROIDCNN-LDP) is pro- posed to realize the methodology for leaf disease detection.

4. A prototype application is built using Python data science plat- form to evaluate the proposed methodology and the underlying algorithms.

The remainder of the paper is structured as follows. Section 2 re- views literature on deep learning methods for leaf disease detec- tion. Section 3 covers the methodology used for the plant disease detection using advanced convolutional neural networks with re- gion of interest awareness. Section 4 presents experimental results and Section 5 concludes the paper and gives directions for future scope of the research.

Related work

This section reviews literature on deep learning models for leaf disease identification. It focuses on neural network models, deep CNN models and pre-trained deep CNN models.

Neural Network (NN) Models

Neural network models are good for processing hyper spectral data. Made a review of different models such as Back-Propagation Neural Network (BPNN), Feed-Forward Neural Network (FFNN) and Generalized Regression Neural Network (GRNN) and their role in early detection of leaf diseases in agriculture [4].

Deep CNN Models

Deep CNN models are widely used for image processing based approach for leaf disease identification. Proposed a methodolo- gy based on deep CNN for identification of rice diseases [3]. The CNN architecture is made up of different layers appropriately con- figured in order to have better performance. In future, they intend to extend their model for fault detection focused on DenseNet-121 for apple leaf disease identification [7]. Multi-label classification approach is used to have more flexible identification of diseases. The diseases thus identified include “Healthy Apple, General Ap- ple Scab, Serious Apple Scab, Apple Gray Spot, General Cedar Apple Rust and Serious Cedar Apple Rust”. Proposed a deep CNN model by modifying LeNet for maize leaf disease classification [8]. The convolutional layers, pooling layers and fully connect- ed layers are configured in such way that their method could im- prove performance. proposed CNN based deep learning model for sugarcane disease identification [9]. Their methodology includes pre-processing, feature extraction and classification. In future, they intend to apply different learning rates to evaluate their meth- od further. Proposed apple leaf disease prediction model based on improved deep CNN [10]. Their real time approach has both dis- ease localization and classification. They also experimented with GoogleNet inception structure along with Rainbow concatenation for disease detection.

Exploited Learning Vector Quantization (LVQ) and deep CNN model for plant leaf disease detection [11]. The usage of LVQ as part of the CNN architecture could leverage classification accura- cy. They intend to improve it with different size of convolutions and different filters. Proposed a deep CNN model with nine lay- ers and six kinds of data augmentation methods are employed in- cluding Principal Component Analysis (PCA) [12]. They intend to improve it to the study of disease using flowers and stem also in future. Used CNN model such as LeNet for leaf disease pre- diction in Tomato crop [14]. Their methodology includes data ac- quisition, data pre-processing and classification. Studied different deep learning models in the usage of plant disease prediction [21]. They could different factors that affect deep learning models. They include disorders with same symptoms, simultaneous disorders, symptom variations, symptom segmentation, conditions of image capture, image background, covariate shift and symptom represen- tations [21.] studied on deep CNN models for leaf image classifi- cation. Focused on deep learning models for disease identification in terms of severity estimation [24]. Francis and investigated on different operations involved in deep CNN based model for leaf disease detection [25]. They classified them into two phases such as feature extraction phase and classification phase.

Proposed an automatic approach for leaf image capture and iden- tification of disease based on CNN models [27]. They built a tool named automaton with GUI for visualization of the detection pro- cess. Used CNN and Conditional Random Field (CRF) approach towards plant disease detection [29]. Proposed an attention em- bedded residual CNN for tomato leaf disease detection [30]. They also proposed a method known as Residual Progressive Feature Extraction (RPFE) for improving quality of training and result in higher level of accuracy. Proposed an improved CNN for real time detection of plant diseases [32]. Their model is named as DR- IACNN which has better feature extraction capability to influence accuracy.

Investigated on deep CNN model for disease detection in plants [5]. Focused on real time plant disease detection using deep learn- ing models [15]. Focused on improved deep CNN models for maize leaf disease detection [16]. proposed a deep learning archi- tecture known as EfficientNet for plant leaf disease classicisation [34]. Explored deep learning and machine learning methods for plant species classification [17]. They found that deep learning models could perform well comparatively. Proposed CNN mod- els for plant disease identification while reviewed CNN models for different plant disease detection [18, 19]. Focused on Apple diseases using multi-scale feature fusion and sub-class categoriza- tion using ResNet model. Singh and proposed CNN based model for crop disease identification [26, 31]. Proposed a deep learning based model known as R-CNN comprising of joint segmentation and leaf skeleton identification [33]. Proposed discriminative CNNs for image classification for plant disease diagnosis [20].

Pre-Trained Deep Learning Models

Pre-trained deep learning models help in reusing knowledge mod- els in leaf disease detection. Explored two pre-trained deep models such as VGG16 net and AlexNet [1]. They used PlantVillege data- set for tomato crop disease prediction. VGG16 net showed bet- ter performance over AlexNet in terms of accuracy. However, in terms of computational load AlexNet showed better performance. Investigated on deep learning models such as ResNet, Inception V4 and VGG16 [2]. The models are evaluated with different layers and also DenseNets. It is observed that the usage of DenseNets consistently increased performance. However, they took more computational time. Studied VGG16 along with SVM model for detection of tomato diseases based on leaf images [6]. With fine- tuned experiments, they could achieve better performance. Used deep learning models such as VGG and AlexNetOWTBn for plant disease detection. They found the potential of such models in the research carried out [13].

Investigated on CNN based deep learning models like VGG16, VGG19 and Inception V3 for leaf disease detection [28]. VGG16 showed highest accuracy when compared with other deep models and also ML models like RF, SVM and SGD. Proposed a Region of Interest (ROI) based approach along with ResNet-50 CNN for identification of plant leaf diseases [35]. Hernández and Lopez [36] focused on Bayesian deep learning models to know and un- certainty and quantify the same. Proposed a CNN based approach is known as self-attention CNN for crop leaf disease identification to leverage robustness in the process [38]. Proposed an improved faster RCNN model for object detection towards tomato disease detection [39]. They also make use of ROI towards improving per- formance. Used CNN model to have a strategy in Cassava disease detection [39]. However, they intend to improve their method to diagnose multiple co-occurring diseases. Used different deep mod- els like AlexNet, ZFNet, VGG16, and VGG19 for automatic iden- tification of Abaca Bunchy Top disease [22]. From the literature it is understood that there are different approaches in deep learning models and pre-trained CNN models found. However, there is lit- tle research on ROI based approach with CNN. It is the motivation behind the work in this paper.

Proposed Methodology

This section presents the proposed methodology of detecting leaf diseases using advanced CNN with region of interest awareness.

The Framework

The methodology for effective leaf disease detection is based on deep learning models and extraction of ROIs. Deep CNN model is configured along with an algorithm for extracting region of inter- est. It results in the realization of plan leaf disease prediction mod- el. With transfer learning, the framework is optimized in training and testing phases. PlantVillege dataset is used in order to evaluate the proposed model. The framework is as presented in Figure 1.

Figure 1: Overview of the proposed methodology

As deep CNN model is found suitable for processing image based inputs like plant leaves, it is used along with the algorithm known as ROI Feature Map Creation (ROI-FMC). This algorithm is re- used in the deep learning process in order to improve the accuracy of prediction.

Algorithm to Extract ROI

ROI Feature Map Creation (ROI-FMC) is the algorithm defined to facilitate quality in disease prediction.

The algorithm ROI-FMC takes a leaf image and ROI maps train- ing set as input and produces ROI feature map for given image. The ROI feature map contains background information, leaf area information and spot area information. The extraction of ROI fea- ture map is based on training ROI maps. It is done by a deep CNN model. The output of ROI-FMC is used as input in the leaf disease prediction algorithm provided in the next slide.

Algorithm for Leaf Disease Prediction

ROI based Deep CNN with Transfer Learning for Leaf Disease Prediction (ROIDCNN-LDP) is the mail algorithm used towards leaf disease prediction.

Algorithm 2 takes leaf test image, ROI maps as training set and pre-trained deep CNN models such as AlexNet, DenseNet19, In- ception_v3, ResNet34, Sqeezenet1_1 and VGG13 as input. This algorithm reuses ROI-FMC algorithm to extract feature map from given test image. Then it is subjected to testing with all pre-trained models with transfer learning. The deep learning models could perform well with the proposed methodology

Experimental Results

The proposed framework is implemented and the results of ex- periments are provided in this section. The results include visual inputs, outputs in terms of heatmap and disease detection. Besides, this section evaluates the proposed deep learning based model and compares with the state of the art.

                                                                     Figure 2: Detection of early blight disease

As presented in Figure 2, the input leaf and the detection of early blight disease is visualized. The results also include heatmap in dif- ferent dimensions.

                                                                       Figure 3: Detection of late blight disease

As presented in Figure 3, the input leaf and the detection of late blight disease is visualized. The results also include heatmap in different dimensions.

Figure 4: Detection of early blight disease

As presented in Figure 4, the input leaf and the detection of early blight disease is visualized. The results also include heatmap in dif- ferent dimensions.

           Table 1: Deep learning models and their performance in leaf disease detection

Figure 5: Evaluation time comparison of deep learning models

As presented in Figure 5, the leaf disease detection models are shown in horizontal axis while the vertical axis shows the evalu- ation time in seconds. The evaluation time is measured with and without retraining. With retraining, it is observed that there is per- formance enhancement as it could reduce time taken for evalua- tion.

Figure 6: Accuracy comparison of deep learning models

As presented in Figure 6, the leaf disease detection models are shown in horizontal axis while the vertical axis shows the accuracy percentage. The accuracy is measured with and without retraining. With retraining, it is observed that there is performance enhance- ment as it could improve accuracy. With the proposed methodol- ogy Inception_V3 with transfer learning showed highest perfor- mance with 0.997637223.

Conclusion and Future Work

We proposed a framework that considers extraction of ROI using deep CNN prior to prediction of pre-trained deep learning models such as VGG13, ResNet34, DenseNet19, AlexNet, Sqeezenet1_1 and Inception_v3. An algorithm named ROI Feature Map Creation (ROI-FMC) is defined to extract ROI for given input image. This will be given as input to another algorithm proposed namely ROI based Deep CNN with Transfer Learning for Leaf Disease Predic- tion (ROIDCNN-LDP). The latter is used to predict leaf diseases. PlantVillage dataset is used for empirical study. Deep CNN models could perform well with the proposed methodology.ROI aware- ness included in the architecture led to performance improvement. Particularly accuracy of the models has been improved. With the proposed methodology Inception_V3 with transfer learning showed highest performance with 0.997637223.Therefore, deep CNN models with transfer learning and ROI awareness are found to be suitable for leaf disease detection. However, Inception_v3 is found better among the other models. In future, we intend to investigate further on the deep learning models other than CNN.

References

1. Rangarajan, A. K., Purushothaman, R., & Ramesh, A. (2018). Tomato crop disease classification using pre-trained deep learning algorithm. Procedia computer science, 133, 1040-1047.
2. Too, E. C., Yujian, L., Njuki, S., & Yingchun, L. (2019). A comparative study of fine-tuning deep learning models for plant disease identification. Computers and Electronics in Ag- riculture, 161, 272-279.
3. Lu, Y., Yi, S., Zeng, N., Liu, Y., & Zhang, Y. (2017). Identi- fication of rice diseases using deep convolutional neural net- works. Neurocomputing, 267, 378-384.
4. Golhani, K., Balasundram, S. K., Vadamalai, G., & Pradhan,B. (2018). A review of neural networks in plant disease de- tection using hyperspectral data. Information Processing in Agriculture, 5(3), 354-371.
5. Sharath, D. M., Kumar, S. A., Rohan, M. G., Suresh, K. V., & Prathap, C. (2020, August). Disease detection in plants us- ing convolutional neural network. In 2020 Third Internation- al Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 389-394). IEEE.
6. Shijie, J., Peiyi, J., & Siping, H. (2017, October). Automatic detection of tomato diseases and pests based on leaf imag- es. In 2017 Chinese automation congress (CAC) (pp. 2537- 2510). IEEE.
7. Zhong, Y., & Zhao, M. (2020). Research on deep learning in apple leaf disease recognition. Computers and Electronics in Agriculture, 168, 105146.
8. Ahila Priyadharshini, R., Arivazhagan, S., Arun, M., & Mir- nalini, A. (2019). Maize leaf disease classification using deep convolutional neural networks. Neural Computing and Appli- cations, 31(12), 8887-8895.
9. Militante, S. V., Gerardo, B. D., & Medina, R. P. (2019, Oc- tober). Sugarcane disease recognition using deep learning. In 2019 IEEE Eurasia conference on IOT, communication and engineering (ECICE) (pp. 575-578). IEEE.
10. Jiang, P., Chen, Y., Liu, B., He, D., & Liang, C. (2019). Re- al-time detection of apple leaf diseases using deep learning approach based on improved convolutional neural networks. IEEE Access, 7, 59069-59080.
11. Geetharamani, G., & Pandian, A. (2019). Identification of plant leaf diseases using a nine-layer deep convolutional neu- ral network. Computers & Electrical Engineering, 76, 323- 338.
12. Sardogan, M., Tuncer, A., & Ozen, Y. (2018, September). Plant leaf disease detection and classification based on CNN with LVQ algorithm. In 2018 3rd international conference on computer science and engineering (UBMK) (pp. 382-385). IEEE.
13. Ferentinos, K. P. (2018). Deep learning models for plant dis- ease detection and diagnosis. Computers and electronics in agriculture, 145, 311-318.
14. Tm, P., Pranathi, A., SaiAshritha, K., Chittaragi, N. B., & Koolagudi, S. G. (2018, August). Tomato leaf disease detec- tion using convolutional neural networks. In 2018 eleventh international conference on contemporary computing (IC3) (pp. 1-5). IEEE.
15. Cap, H. Q., Suwa, K., Fujita, E., Kagiwada, S., Uga, H., & Iyatomi, H. (2018, March). A deep learning approach for on- site plant leaf detection. In 2018 IEEE 14th International Col- loquium on Signal Processing & Its Applications (CSPA) (pp. 118-122). IEEE.
16. Zhang, X., Qiao, Y., Meng, F., Fan, C., & Zhang, M. (2018). Identification of maize leaf diseases using improved deep con- volutional neural networks. Ieee Access, 6, 30370-30377.
17. Wei Tan, J., Chang, S. W., Abdul-Kareem, S., Yap, H. J., & Yong, K. T. (2018). Deep learning for plant species classifi- cation using leaf vein morphometric. IEEE/ACM transactions on computational biology and bioinformatics, 17(1), 82-90.
18. Yadav, S., Sengar, N., Singh, A., Singh, A., & Dutta, M. K. (2021). Identification of disease using deep learning and eval- uation of bacteriosis in peach leaf. Ecological Informatics, 61, 101247.
19. Li, L., Zhang, S., & Wang, B. (2021). Plant disease detection and classification by deep learning—a review. IEEE Access, 9, 56683-56698.
20. Tavakoli, H., Alirezazadeh, P., Hedayatipour, A., Nasib, A. B., & Landwehr, N. (2021). Leaf image-based classification of some common bean cultivars using discriminative convolu- tional neural networks. Computers and electronics in agricul- ture, 181, 105935.
21. Barbedo, J. G. (2018). Factors influencing the use of deep learning for plant disease recognition. Biosystems engineer- ing, 172, 84-91.
22. Patayon, U. B., & Crisostomo, R. V. (2021). Automatic identi- fication of abaca bunchy top disease using deep learning mod- els. Procedia Computer Science, 179, 321-329.
23. Sladojevic, S., Arsenovic, M., Anderla, A., Culibrk, D., & Ste- fanovic, D. (2016). Deep neural networks based recognition of plant diseases by leaf image classification. Computationalintelligence and neuroscience, 2016.
24. Wang, G., Sun, Y., & Wang, J. (2017). Automatic image-based plant disease severity estimation using deep learning. Compu- tational intelligence and neuroscience, 2017.
25. Francis, M., & Deisy, C. (2019, March). Disease detection and classification in agricultural plants using convolutional neural networks—a visual understanding. In 2019 6th Internation- al Conference on Signal Processing and Integrated Networks (SPIN) (pp. 1063-1068). IEEE.
26. Luo, Y., Sun, J., Shen, J., Wu, X., Wang, L., & Zhu, W. (2021).Apple leaf disease recognition and sub-class categorization based on improved multi-scale feature fusion network. IEEE Access, 9, 95517-95527.
27. De Luna, R. G., Dadios, E. P., & Bandala, A. A. (2018, Oc- tober). Automated image capturing system for deep learn- ing-based tomato plant leaf disease detection and recognition. In TENCON 2018-2018 IEEE Region 10 Conference (pp. 1414-1419). IEEE.
28. Sujatha, R., Chatterjee, J. M., Jhanjhi, N. Z., & Brohi, S. N. (2021). Performance of deep learning vs machine learning in plant leaf disease detection. Microprocessors and Microsys- tems, 80, 103615.
29. Wiesner-Hanks, T., Wu, H., Stewart, E., DeChant, C., Kaczmar, N., Lipson, H., ... & Nelson, R. J. (2019). Milli- meter-level plant disease detection from aerial photographs via deep learning and crowdsourced data. Frontiers in Plant Science, 10, 1550.
30. Karthik, R., Hariharan, M., Anand, S., Mathikshara, P., John- son, A., & Menaka, R. (2020). Attention embedded residu- al CNN for disease detection in tomato leaves. Applied Soft Computing, 86, 105933.
31. Singh, N., & Sabrol, H. (2021). Convolutional neural net- works-an extensive arena of deep learning. A comprehensive study. Archives of Computational Methods in Engineering, 28(7), 4755-4780.
32. Xie, X., Ma, Y., Liu, B., He, J., Li, S., & Wang, H. (2020). A deep-learning-based real-time detector for grape leaf diseases using improved convolutional neural networks. Frontiers in plant science, 11, 751.
33. ‘Zhang, L., Xia, C., Xiao, D., Weckler, P., Lan, Y., & Lee, J.M. (2021). A coarse-to-fine leaf detection approach based on leaf skeleton identification and joint segmentation. biosyste- ms engineering, 206, 94-108.
34. Atila, Ü., Uçar, M., Akyol, K., & Uçar, E. (2021). Plant leaf disease classification using EfficientNet deep learning model. Ecological Informatics, 61, 101182.
35. Leong, K. K., & Tze, L. L. (2020, June). Plant leaf diseases identification using convolutional neural network with treat- ment handling system. In 2020 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS) (pp. 39-44). IEEE.
36. Hernández, S., & López, J. L. (2020). Uncertainty quantifica- tion for plant disease detection using Bayesian deep learning. Applied Soft Computing, 96, 106597.
37. Zeng, W., & Li, M. (2020). Crop leaf disease recognition based on Self-Attention convolutional neural network. Com- puters and electronics in agriculture, 172, 105341.
38. Zhang, Y., Song, C., & Zhang, D. (2020). Deep learning-based object detection improvement for tomato disease. IEEE Ac- cess, 8, 56607-56614.
39. Sambasivam, G., & Opiyo, G. D. (2021). A predictive ma- chine learning application in agriculture: Cassava disease detection and classification with imbalanced dataset using convolutional neural networks. Egyptian Informatics Journal, 22(1), 27-34.