Fruit type and weight recognition for payment purposes using computer vision

Abstract

Nowadays, retailers and supermarkets are still selling fruit manually, employees must memorize fruit types, scale weight, and then print out a QR (quick response) code containing price and other details. Having to remember every type of fruit will lead to sellers having some mistakes and giving the wrong estimation. In this work, a novel solution is proposed to solve that problem using computer vision together with the connectivity of digital scales and a webcam. The objective is to make the process of giving prices be done with minimum human effort. To achieve that, Computer vision is used in checkout systems for scanning fruit. The idea of this work is to calculate the price of fruit by recognizing the fruit categories by scanning one or multiple objects of the same type in a specific condition and together with weight value from the scale. Fruit recognition in retail is challenging work due to features such as intensity, color, shape, and texture. To predict fruit, a CNN network is used with DenseNet201, the purpose is to make the application process faster. To process the weight directly, an IoT weight sensor, which can connect and send values to other devices namely computers, is needed in combination with a camera for identifying purposes. Unfortunately, such a type of scale is not available in the common marketplace and we have to build our own using a load cell, an HX711 module, and an Arduino Uno. The model reached the best accuracy score of 99.62% and the application takes around 300ms for the result.