Deep Learning Approaches to Segment Eucalyptus Tree Images

Abstract

Agribusiness is one of Brazil's primary sources of wealth and employment, representing a significant portion of the national Gross Domestic Product (GDP). In 2021, the agribusiness sector reached 27.4% of the Brazilian GDP, the highest share since 2004, when it reached 27.53%. The forest-based industry is an important segment of agribusiness, as it provides vital inputs for various industrial sectors, such as wood products, furniture, and paper. Planted forests play an essential role in carbon capture and other ecosystem services, with eucalyptus being the most used tree, with 7.3 million hectares of eucalyptus forests in 2021. Tree mapping is vital for the economy and environment, and artificial intelligence-based solutions are valuable decision support tools in agriculture and tree mapping. Consequently, there is a strong incentive to look for more comprehensive solutions that use advanced deep learning technologies for this area. Thus, this work aims to evaluate efficient deep learning convolutional neural networks for image segmentation of eucalyptus trunks and present a specific segmentation proposal for eucalyptus trunks that can benefit agricultural applications or decision support tools for tree mapping. This work was divided into two main steps to evaluate the segmentation networks and create a post-processing technique. The first stage of this study evaluated the efficiency of deep learning networks in the semantic segmentation of eucalyptus trunks in panoramic images in RGB colors captured at ground level. The deep learning networks FCN, GCNet, ANN, and PointRend were evaluated in this step for image segmentation of eucalyptus trunks. Training and evaluation of the networks were performed using a five-step cross-validation approach, using a dataset composed of manually annotated images of a eucalyptus forest. The initial dataset was created using a spherical field of view camera. It included a variety of eucalyptus trees with distinct characteristics, such as variations in distances between trunks and changes in curvature, sizes, and diameters of trunks, which pose significant challenges for deep learning methods in semantic segmentation tasks. For the first stage of this study, the FCN model presented the best performance, with pixel precision of 78.87% and mIoU of 70.06%, in addition to obtaining a good inference time. The GCNet and ANN networks also performed similarly to the FCN but with negative impacts on their ability to generalize tasks in specific contexts. The study concludes that the FCN was the most robust, among the evaluated methods, for semantic segmentation of images of trees in panoramic images. This assessment of segmentation networks can be a crucial step toward developing other relevant tools in forest management, such as estimating trunk height and diameter. The second step of this work was to create and evaluate a post-processing technique for RGB-D images to improve the results of current semantic networks for segmentation in eucalyptus images. We created a new dataset image using images obtained from a stereo camera, which captured not only the color information (RGB) but also the depth information, which allowed an even more complete view of the eucalyptus forest. After the construction of the new image bank, its annotation was carried out by specialists. The next stage of this study was the evaluation of six image semantic segmentation networks and the comparison with results before and after applying the post-processing technique. We trained, evaluated, and tested the FCN, ANN, GCNet, SETR, SegFormer, and DPT networks on the annotated images. The post-processing technique significantly improved the results of the tested image segmentation networks, with a significant gain of 24.13% in IoU and 13.11% in F1-score for convolution-based networks and 12.49% for IoU and 6.56% in F1-score for transformer-based networks. The SegFormer network obtained the best results in all tests before and after applying the technique. The technique also effectively corrected segmentation flaws, erosion, and dilation errors, resulting in more accurate edges and better-delimited trunks. The average computational cost of the technique was 0.019 seconds, indicating that it can be applied in segmentation networks without compromising performance. The results obtained by applying the post-processing technique propose an innovative approach with low computational cost and significant improvements to existing segmentation networks.