Extract Information from Vietnamese ID Card Images

Abstract

The efficient extraction of information from ID cards is vital for various daily services, such as legal, banking, insurance, and medical processes. Nevertheless, in numerous developing countries nations like Vietnam, this task is predominantly manual, resulting in time-consuming, monotonous, and error-prone processes. This thesis presents a deep learning system specifically designed to extract information from images of Vietnamese ID cards. The proposed system involves three sequential steps: ID card alignment algorithm, text detection and text recognition. The initial step incorporates two neural networks, YOLACT and ResNet50 for segmentation and classification model, alongside an image processing technique. The second step employs YOLOv7 for text detection. The third step is to utilize VietOCR based on Attention OCR for recognizing Vietnamese optical text on the cards. In experimental evaluations, the proposed system demonstrates a notable reduction in processing time, especially in scenarios where corners of the ID card are obstructed. It effectively addresses challenges when compared to existing methodologies, achieving a high mAP@[0.5:0.95] for box is 97.21 % and mask is 99.58 % for ID card segmentation, 74.0 % for text detection. The system also exhibits exceptional precision score for full-sentence with recognition rates of 98.19 % for Vietnamese optical texts and 97.60 % for the ID card classification model. Overall, our implementation of the proposed method achieves an average system-wide accuracy rate about 97.98 %.