Fast and accurate detection of ripe tomatoes on plant, which replaces manual labor with a robotic vision-based harvesting system, is a challenging task. Tomatoes in adjacent positions are easily mistaken as a single tomato by image recognition methods. In this study, a ripe tomato detection method that combines deep learning with edge contour detection is proposed. Our approach efficiently separates target tomatoes from overlapping tomatoes to detect individual fruits. This approach yields several improvements. First, deep learning requires less time and extracts deeper features than traditional methods for assessing candidate ripe tomato regions. Second, we use Gaussian density function of H and S in the HSV color space to help segment tomato regions from the background, followed by erosion and dilation on the tomato body to separate adjacent tomatoes and remove peripheral subpixels from all detected ripe tomatoes. Third, an adaptive threshold intuitionistic fuzzy set (IFS) method was developed to identify the tomato’s edge, and it performs well in detecting blurred edges in overlapping regions. To improve the efficiency and stability of edge detection under natural conditions, we adopted an illumination adjustment algorithm for the tomato image before edge detection. As samples, we collected images showing tomatoes that were separated, adjacent, overlapped, and even shaded by leaves. The widths and heights of these tomato samples were calculated and analyzed to evaluate the detection performance of the proposed method. The root mean square error (RMSE) results for tomato width and height using the proposed method are 2.996 pixels and 3.306 pixels, respectively. The mean relative error percent (MRE%) values for horizontal and vertical center position shift are 0.261% and 1.179%, respectively. These results demonstrate that the proposed method improves tomato detection accuracy and that it can be further applied in the harvesting process of agricultural robots.

Introduction

Tomatoes are one of the most important and popular fruit crops. Tomatoes offer humans many essential and beneficial nutrients such as antioxidants and vitamins C and A. As tomato demand increases, tomatoes are increasingly grown in greenhouses. However, manual harvesting is time consuming and costly, and as China’s labor costs rise, the adoption of agricultural automation processes is inevitable. Such processes are of great significance for reducing agriculture labor costs and improving a country’s industrial structure. Therefore, it is necessary to develop automatic tomato pickers. Although most agricultural robots— fruit harvesting systems in particular—use computer vision to detect fruit targets, accurate fruit detection is a challenging research topic. It is difficult to develop a vision system that functions as intelligently as a human and can easily identify fruit, especially in the presence of overlapping fruits or large leaf occlusions. The performance of the robot’s visual system directly affects tomato picking and operational safety. Improving the recognition rate of the visual system can increase the locating accuracy of the robot arm. In this study, we mainly aimed to identify ripe tomatoes based on a vision system. Systems designed to count or harvest fruit require accurate detection schemes that can overcome challenges such as naturally occurring changes in illumination, shape, pose, color and viewpoint.