Semantic Segmentation of Land Use

The full text together with a short presentation is available in this release.

Abstract

The automated identification of emergency landing fields is a complex challenge which requires to process huge quantities of information. This thesis explores the use of convolutional neural networks to perform a classification of land use through semantic segmentation to support the identification process. For that, three popular reference architectures of U-Net, FC-DenseNet and W-Net are implemented and applied to this challenge. The experiments show that U-Net and FC-DenseNet achieve adequate segmentation results, while the unsupervised learning process of W-Net fails to learn a proper class differentiation. In a second step, several spectral vegetation indices are investigated whether they are applicable to further narrow down the number of suitable emergency landing fields. It is demonstrated that with the given dataset, the indices do not provide any meaningful information. Overall, this thesis offers a valuable contribution to the improvement of the automatic identification of emergency landing fields.

Segmetation Results

Original Image	Ground Truth	U-Net	FC-DenseNet

References

1. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. 2015. [Online].
2. P. Arbeláez, M. Maire, C. Fowlkes, J. Malik. Contour Detection and Hierarchical Image Segmentation. IEEE TPAMI: vol. 33, no. 5, pp. 898 - 916, 2011. DOI: 10.1109/TPAMI.2010.161.
3. I. Arganda-Carreras, S. Seung, A. Cardona, J. Schindelin. ISBI Challenge: Segmentation of neuronal structures in EM stacks. 2012. [Online].
4. I. Arganda-Carreras, S. C. Turaga, D. R. Berger, D. Ciresan, A. Giusti. Crowdsourcing the creation of image segmentation algorithms for connectomics. Front. Neuroanat: vol. 9, no. 142. 2015. DOI: 10.3389/fnana.2015.00142.
5. G. J. Brostow, J. Shotton, J. Fauqueur, R. Cipolla. Segmentation and Recognition Using Structure from Motion Point Clouds. In ECCV: pp. 44 - 57, 2008.
6. J. Canny. A Computational Approach to Edge Detection. IEEE TPAMI: vol. 8, no. 6, pp. 679 - 698. 1986. DOI: 10.1109/TPAMI.1986.4767851.
7. L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. 2017. arXiv:1606.00915.
8. F. Chollet. Xception: Deep Learning with Depthwise Separable Convolutions. 2017. arXiv:1610.02357.
9. Y. Cui, M. Jia, T. Lin, Y. Song, S. Belongie. Class-Balanced Loss Based on Effective Number of Samples. 2019. arXiv:1901.05555.
10. J. Deng, W. Dong, R. Socher, L. Li, K. Li, L. Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR: pp. 248 - 255, 2009. DOI: 10.1109/CVPR.2009.5206848
11. L. Deng, D. Yu. Deep Learning: Methods and Applications. FTSP: vol. 7, pp. 197 - 387, 2014. DOI: 10.1561/2000000039
12. V. Dumoulin, F. Visin. A guide to convolution arithmetic for deep learning. 2018. arXiv:1603.07285.
13. M. Everingham, L. V. Gool, C. K. I. Williams, J. Winn, A. Zisserman. The PASCAL Visual Object Classes (VOC) Challenge. 2012. [Online].
14. M. Everingham, S. M. A. Eslami, L. V. Gool, C. K. I. Williams, J. Winn, A. Zisserman. The Pascal Visual Object Classes Challenge: A Retrospective. IJCV: vol. 111, pp. 98 - 136, 2015. DOI: 10.1007/s11263-014-0733-5
15. A. Géron. Praxiseinstieg: Machine Learning mit Scikit-Learn und TensorFlow. O'Reilly dpunkt.verlag, Heidelberg. 2017. Translated by K. Rother. ISBN 978-3-96009-061-8.
16. R. Girshick, J. Donahue, T. Darrell, J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. 2014. arXiv:1311.2524.
17. GISGeography. What is NDVI (Normalized Difference Vegetation Index). 2020. [Online].
18. I. Goodfellow, Y. Bengio, A. Courville. Deep Learning. MIT Press. 2016. [Online].
19. FernUniversität in Hagen. Emergency Landing Field Identification (ELFI). 2020. [Online].
20. FernUniversität in Hagen. Flugassistenzsysteme - Forschungsbereich Notlandeassistenzsysteme. 2020. [Online].
21. H. He, E. A. Garcia. Learning from Imbalanced Data. IEEE TKDE: vol. 21, no. 9, pp. 1263 - 1284, 2009. DOI: 10.1109/TKDE.2008.239
22. K. He, G. Gkioxari, P. Dollár, R. Girshick. Mask R-CNN. 2018. arXiv:1703.06870.
23. G. Hinton, N. Srivastava, K. Swersky. rmsprop: Divide the gradient by a running average of its recent magnitude. Lecture Notes. 2014. [Online].
24. N. Horning, D. C. Russell. Global Land Vegetation - An Electronic Textbook. 2003. [Online].
25. G. Huang, Z. Liu, L. v. d. Maaten, K. Q. Weinberger. Densely Connected Convolutional Networks. 2018. arXiv:1608.06993.
26. A. Huete. A soil-adjusted vegetation index (SAVI). Remote Sensing of Environment: vol. 25, no. 3, pp. 295 - 309, 1988. DOI: 10.1016/0034-4257(88)90106-X
27. A. Huete, K. Didan, T. Miura, E. P. Rodriguez, X. Gao, L. G. Ferreira. Overview of the radiometric and biophysical performance of the MODIS vegetation indices. Remote Sensing of Environment: vol. 83, no. 1, pp. 195 - 213, 2002. DOI: 10.1016/S0034-4257(02)00096-2
28. Ministerium des Innern des Landes Nordrhein-Westfalen. GEOportal.NRW. [Online].
29. S. Jégou, M. Drozdzal, D. Vazquez, A. Romero, Y. Bengio. The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation. 2017. arXiv:1611.09326.
30. Bundesamt für Kartographie und Geodäsie. Digital Basic Landscape Model (Basic-DLM). 2020. [Online].
31. M. Klein, A. Klos, W. Schiffmann. A Smart Flight Director for Emergency Landings with Dynamical Recalculation of Stable Glide Paths. AIAA Forum. 2020. DOI: 10.2514/6.2020-3098.
32. A. Klos, J. Lenhardt, M. Klein, W. Schiffmann. Multi-Modal Image Processing Pipeline for a Reliable Emergency Landing Field Identification. CEAS GNC. 2019. [Online].
33. A. Klos, M. Rosenbaum, W. Schiffmann. Ensemble Transfer Learning for Emergency Landing Field Identification on Moderate Resource Heterogeneous Kubernetes Cluster. ISCNC. 2020. arxiv:2006.14887.
34. K. Köhler. Semantic Image Segmentation - Code for Preprocessing, Model Training and Deployment. 2020. DOI: 10.5281/zenodo.4065143
35. Bezirksregierung Köln. TIM-Online. [Online].
36. A. Krizhevsky, I. Sutskever, G. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. NIPS: vol. 25, 2012. DOI: 10.1145/3065386.
37. Y. LeCun, C. Cortes. MNIST handwritten digit database. 2010. [Online].
38. T. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, P. Dollár. Microsoft COCO: Common Objects in Context. 2015. arXiv:1405.0312.
39. Google LLC. TensorFlow - Case Studies and Mentions. [Online].
40. Google LLC. TensorFlow - Serving Models. [Online].
41. J. Long, E. Shelhamer, T. Darrell. Fully Convolutional Networks for Semantic Segmentation. 2015. arXiv:1411.4038.
42. S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, D. Terzopoulos. Image Segmentation Using Deep Learning: A Survey. 2020. arXiv:2001.05566.
43. Y. Nesterov. A method of solving a convex programming problem with convergence rate O(1/sqrt(k)). Soviet Mathematics Doklady: vol. 27, pp. 372 - 376, 1983.
44. Bezirksregierung Köln (Abteilung Geobasis NRW). Topographische Bildinformationen - Luftbildmaterial von Nordrhein-Westfalen. 2016. Brochure with Product Information.
45. C. Nwankpa, W. Ijomah, A. Gachagan, S. Marshall. Activation Functions: Comparison of trends in Practice and Research for Deep Learning. 2018. arXiv:1811.03378.
46. G. Papandreou, L. Chen, K. Murphy, A. L. Yuille. Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation. 2015. arXiv:1502.02734.
47. J. Qi, A. Chehbouni, A. Huete, Y. Kerr, S. Sorooshian. A Modified Soil Adjusted Vegetation Index. Remote Sensing of Environment: vol. 48, no. 2, pp. 119 - 126, 1994. DOI: 10.1016/0034-4257(94)90134-1
48. Sh. Ren, K. He, R. Girshick, J. Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. 2016. arXiv:1506.01497.
49. R. Rojas. Neural Networks - A Systematic Introduction. Springer-Verlag, Berlin. 1996. ISBN 978-3-642-61068-4.
50. O. Ronneberger, P. Fischer, T. Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. 2015. arXiv:1505.04597.
51. Ch. Szegedy, A. Toshev, D. Erhan. Deep Neural Networks for Object Detection. NIPS Proceedings: vol. 26, pp. 2553 - 2561, 2013. DOI: 10.5555/2999792.2999897
52. E. Tiu. Metrics to Evaluate your Semantic Segmentation Model. 2019. [Online].
53. Stanford University. Convolutional Neural Networks for Visual Recognition. Lecture Notes. 2020. [Online].
54. J. Weier, D. Herring. Measuring Vegetation (NDVI & EVI). 2000. [Online].
55. X. Xia, B. Kulis. W-Net: A Deep Model for Fully Unsupervised Image Segmentation. 2017. arXiv:1711.08506.
56. Y. Xu, R. Goodacre. On Splitting Training and Validation Set: A Comparative Study of Cross-Validation, Bootstrap and Systematic Sampling for Estimating the Generalization Performance of Supervised Learning. Journal of Analysis and Testing: vol. 2, pp. 249 - 262, 2018. DOI: 10.1007/s41664-018-0068-2.