Real-time detection and localization of ureteral openings in urological endoscopy and surgical videos
In urological endoscopy and surgery, the detection and localization of ureteral opening is very important. However, because the appearance of the ureteral opening varies from individual to individual, from time to time, and from different pathological factors, it is sometimes challenging to accurately locate and locate the ureteral opening. To automatically identify different types of ureteral openings in surgical videos, this paper proposes a deep learning-based ureteral opening detection and tracking system. The framework is mainly composed of three components: the preprocessing part, the ureter opening detection model, and the tracking model. For the preprocessing part, this paper applies general data augmentation strategy and specific data augmentation strategy to increase the diversity of training samples. The ureter opening detection model (Refined-SSD) is obtained by improving the classic model Single Shot Multi Box Detector (SSD) in the field of target detection. Then Refined-SSD was fused with the tracking algorithm CSRT to form the ureteral opening detection and tracking system. In this paper, we only use resectoscope images with more complex background information to train the detection model, and then use ureteroscope images for testing. The experimental results prove that the model trained with resectoscope images can be successfully applied to other types of urological endoscope images, and its evaluation indexes are all around 0.9. We further evaluate the proposed detection model on the resectoscope video and ureteroscope video datasets, and the experiments show that the proposed ureteral opening detection model can identify and localize the ureteral opening in two different uroscopes in real-time in the video. . In addition, in resectoscope video sequences and ureteroscope video sequences, we not only compared the performance of the proposed detection and tracking model (Refined-SSD+CSRT) with that of a single detection model, but also fused with other detection models. The effects of four tracking algorithms are compared, and the experiments show that the ureter opening detection and tracking model proposed in this paper has superior performance and achieves an average detection speed of 20ms per frame. Therefore, the detection and tracking model can accurately and real-time identify and locate ureteral openings in uroscopy surgery videos, and it can be applied to different types of uroscopy
