Automated recognition of surgical phases in robot-assisted radical prostatectomy using a Hiera-based vision transformer.
Source: PubMed, NCBI / U.S. National Library of Medicine
To construct a reliable robot-assisted radical prostatectomy (RARP) surgical phase-recognition model and sought to deploy the model within the SurgSmart platform (Chengdu Withai Innovations Technology Co., Ltd., Chengdu, China) to explore its feasibility and preliminary clinical utility in real surgical workflows, including its usability, interpretability, and potential value for surgical education and quality review. A total of 72 complete RARP procedure videos were collected, and divided into training, verification and test groups (7:1:2). We adopted Hiera (Meta Platforms, Inc., Menlo Park, CA, USA), a hierarchical vision transformer, as the backbone model for surgical phase recognition. Model performance was evaluated using precision, recall, F1-score, and overall accuracy. The trained model was deployed on the SurgSmart platform and tested in two real world RARP procedures to evaluate its feasibility for intraoperative and postoperative use. Temporal annotation quality was high, with a mean inter-annotator Intersection over Union (IoU) score of 0.99 and a weighted IoU score of 0.97. Based on the finalised annotations, the trained Hiera model achieved a weighted F1-score of 0.91, a macro F1-score of 0.90, and an overall accuracy of 0.91 on the test set. Across the seven surgical phases, 'urethral anastomosis' and 'intrafascial dissection' reached F1-scores of 0.96 and 0.92, and the remaining phases demonstrated F1-scores within the 0.85-0.96 range. The confusion matrix dem
Abstract
To construct a reliable robot-assisted radical prostatectomy (RARP) surgical phase-recognition model and sought to deploy the model within the SurgSmart platform (Chengdu Withai Innovations Technology Co., Ltd., Chengdu, China) to explore its feasibility and preliminary clinical utility in real surgical workflows, including its usability, interpretability, and potential value for surgical education and quality review. A total of 72 complete RARP procedure videos were collected, and divided into training, verification and test groups (7:1:2). We adopted Hiera (Meta Platforms, Inc., Menlo Park, CA, USA), a hierarchical vision transformer, as the backbone model for surgical phase recognition. Model performance was evaluated using precision, recall, F1-score, and overall accuracy. The trained model was deployed on the SurgSmart platform and tested in two real world RARP procedures to evaluate its feasibility for intraoperative and postoperative use. Temporal annotation quality was high, with a mean inter-annotator Intersection over Union (IoU) score of 0.99 and a weighted IoU score of 0.97. Based on the finalised annotations, the trained Hiera model achieved a weighted F1-score of 0.91, a macro F1-score of 0.90, and an overall accuracy of 0.91 on the test set. Across the seven surgical phases, 'urethral anastomosis' and 'intrafascial dissection' reached F1-scores of 0.96 and 0.92, and the remaining phases demonstrated F1-scores within the 0.85-0.96 range. The confusion matrix demonstrated that most surgical phases were correctly classified, with a high concentration of samples along the diagonal cells, indicating strong alignment between predicted and ground-truth labels. During real-time deployment, the system processed live surgical video streams continuously and generated phase predictions throughout the operation without interruption. All outputs were produced without processing errors or interface interruptions, confirming stable operation in both real-time and retrospective modes. Accurate surgical phase recognition might be achievable in RARP under a limited but high-quality data setting, although further validation with larger and more diverse datasets is needed to confirm these findings.