Automatic Camera Extrinsics Estimation in the Catheterization Laboratory

Automatic Camera Extrinsics Estimation in the Catheterization Laboratory

Zeng, Jinchen (TU Delft Electrical Engineering, Mathematics and Computer Science)

Dauwels, J.H.G. (mentor)
Zhang, X. (mentor)

Delft University of Technology

Electrical Engineering | Circuits and Systems

2022-09-26

Surgical workflow analysis has gained more importance in operating rooms, which could take responsibility for the working condition, the safety of both patients and surgical personnel, as well as the working efficiency. Focusing on the optimization of the workflow, a set of cameras is installed in the Catheterization Laboratory in Reinier de Graaf Gasthuis for multiple computer vision related researches. However, our cameras are calibrated only once after installation. The orientation and position of the cameras could be changed after days or months, which could lead to a wrong localization.
Compared with the traditional calibration method (calibration patterns or markers), we propose a new image-based camera pose estimation pipeline tested in the Catheterization Laboratory. Our proposed pipeline exploits the object detection model (Scaled-yolov4) to detect fixed objects. The mean average precision with 50\% IoU threshold (mAP@.5) achieves more than 0.99 for all detected objects. Then use a self-supervised key-point detector and descriptor (SuperPoint). With the detected key-points, a feature matching technique based on graph neural networks (SuperGlue) is adopted to match the key-points on the target image with reference points annotated in the image databases (image DBs). The point-correspondences between the image coordinates and the 3D coordinates are applied to solve Perspective-n-Point (PnP) problem to compute the orientation and position of each camera (Camera pose). The final camera pose estimation achieves a 5.79 pixel reprojection error with a 4.97 cm Euclidean distance error.
Compared with other image-based camera pose estimation techniques, our pipeline requires no 3D reconstruction or 3D point cloud in the scene model. Using the video from real procedures, we show that the pipeline can estimate the camera pose with high accuracy.

http://resolver.tudelft.nl/uuid:91f7878b-ef71-47db-90b5-d4b48cefb314

Student theses

master thesis

© 2022 Jinchen Zeng