ABSTRACT
Robotic-assisted prostate biopsy procedures may revolutionize the field of urology by providing improved accuracy, precision, and patient comfort, together with early cancer detection and treatment. This article delves into the integration of artificial intelligence (AI) in the robotic system to further enhance the efficacy and efficiency of these biopsies. AI algorithms are employed for various crucial aspects of the procedure, including target localization, image fusion, needle trajectory planning, and real-time feedback. In this article, we explore the utilization of AI-driven image processing for the identification of the prostate in pre-operative and intra-operative images with the purpose of the automation of the image fusion process, as well as the identification and localization of prostate lesions. The use of a robot and AI aid in creating an intra-operative model of the prostate, facilitating precise biopsy needle placement. Furthermore, real-time AI feedback is used to track the movement of the target and to update the image fusion instantaneously. AI algorithms provide immediate guidance, alerting clinicians to any deviations from the planned trajectory, thus minimizing the risk of complications. We discuss the potential challenges and ethical considerations associated with AI integration in robotic prostate biopsy, including data privacy, transparency, and the importance of maintaining clinician expertise. Additionally, the article highlights ongoing research and development in the field, shedding light on the promising future of AI in prostate biopsy. In conclusion, the integration of artificial intelligence in robotic prostate biopsy is poised to transform the field, optimizing the diagnostic process, and improving patient outcomes. This article offers valuable insights into the current state and future prospects of AI-driven advancements in prostate biopsy procedures.
Key words: Robotics, artificial intelligence, image fusion, medical applications
INTRODUCTION
Prostate cancer is the second most common cancer in men and its diagnosis is triggered either by the increase of the prostate specific antigen (PSA) biomarker in the blood, or by more specific symptoms, after which a multi-parametric magnetic resonance image (mpMRI) is taken to identify lesions with a diameter larger than 5 mm. When prostate cancer is suspected, diagnosis must be confirmed by a biopsy ad subsequent histological analysis. Prostate biopsy has an accuracy lower than 70%.[1] The reason is the bad target identification (cognitive) and bad needle guidance (manual) leading to high personal and societal costs.
Robotics and artificial intelligence (AI) are possible solutions to compensate human errors. The robotic technologies could allow doctors to perform complex tasks with more precision, safety, repeatability, and control. Systems based on AI are also used to support medical procedures, in particular for data/image analysis and processing, but the physical interaction with the patient is carried by human medical practitioners. The powerful merge of robots and AI algorithms has not yet been exploited at the full potential to improve the medical field.
KEY FEATURES
The integration of AI algorithms with a robotic device will overcome the cognitive and manual limitations of a human operator. Most of the cognitive and manual functions carried out by the physician will be replaced by AI algorithms and by robotics motions, while the decisions and the needle insertion will be performed by the doctor.[2] AI algorithms provide guidance for: (1) identification of the prostate boundaries and biopsy targets in magnetic resonance images (MRI), (2) fusion of MRI and ultrasound (US) images, (3) acquisition of US images and (4) computation of the optimal needle trajectories and (5) identification of the target lesion to be biopsied. The role of the robot is to (1) compute the correct needle orientation, (2) tracking of the needle by the US probe, (3) move to the entry point (4) adjustment of the US probe and (5) monitor the needle insertion.
CASE STUDY: PROST ROBOT
The PROST robot [3] is an example of how a prostate biopsy robot may integrate AI to improve the procedure. We describe in the following the main components of the robot and the pre-clinical testing.
The mechanical structure is composed of the following elements
(1) A robotic positioner mounted on a support cart that allows gross positioning. This part could be an active or a passive arm. In the case of active movement, the tracking is made by the integrated encoders, while a passive arm should be equipped with position sensors.
(2) A robotic head that determines the needle orientation and the movement of the ultrasound transrectal probe. The solution chosen for this prototype[2] consists of two-joint arms that move along parallel planes (Figure 1). They are coupled by an axis passing through the center of the two joints that determines the needle orientation. The movement of the US probe is rotational along the roll axis that coincides with the probe axis.
Figure 1. Rendering of the mechanical structure of the PROST robot. The system is composed of a support arm that moves the robotic head. The robotic head allows the orientation of the needle and the movement of the US probe. US, ultrasound.
Software structure
(1) A middleware that interfaces with the electronics and the hardware.
(2) A graphical user interface (GUI) that allows to monitor the workflow of the robot so that the doctor retains the complete control of the surgical task. Through PROST’s GUI, the insertion of the needle is monitored in two-dimensional (2D) and 3D, along with the current distance from the target location.
(3) A software module that implements automatic fusion of pre- and intra-operative images, as described in the next section.
According to the autonomy classification described,[4] PROST reaches the autonomy level 2 (task autonomy), while according to[5] the level of autonomy (LoA) is, again, level 2. This level corresponds to degree of autonomy (DoA) 3–5, in the finer classification based on the International Electrotechnical Commission Technical Requirement (IEC/TR) 60601-4-1.
Considering this classification, some subtasks can be placed at DoA 3 i.e. shared decision (e.g. MRI/US segmentation), or DoA 4 i.e. decision support (e.g. MRI/US fusion), or DoA 5 i.e. blended decision (e.g. entry point selection).
PROST is a needle guide system for the perineal access in prostate biopsy; the insertion of the needle is manual and there is no mechanical structure at the skin entry point. The forces exerted on the patient are on the needle. PROST is used only to orient the cannula and the US probe (Figure 1). The rotation of the US probe is the only interaction with the patient’s body directly. The only contact point is the rectum. The safety issues of autonomous operation can be addressed simply by limiting friction and range of motion: By using sufficient gel around the probe, by placing an upper bound on probe rotation speed and by limiting the angular displacement to the range necessary for the prostate and surrounding tissues to be brought into view.
AUTOMATIC AI FUSION FOR PROST ROBOTIC SYSTEM
The pipeline of the PROST robot fusion process starts with the automatic segmentation of the prostate in pre-operative MRI and in real-time US. The MRI is already arranged in a 3D grid of voxels, therefore the algorithm produces 3D segmented data, whereas the 3D reconstruction of the prostate from US images is made using the robotic movement and tracking of the probe.
The convolutional neural network that implements the prostate segmentation PROST-Net has been published.[6,7]
While the robotic scanning of the whole prostate gland allows for an initial alignment, continuous real-time imaging and automatic AI based segmentation in real-time allow prostate tracking during the procedure, compensating the interaction with the US probe and unexpected patient’s movements.
In an initial training of PROST-Net for MRI segmentation, data from the Cancer Imaging Archive website (https://wiki.cancerimagingarchive. net/) was used. The dataset contains transverse relaxation time (T2) MRI scans, 3D reconstructed US and prostate, and Prostate Imaging Reporting and Data System (PIRADS)≥ 3 lesions segmentation in both MRI and US.
Subsequently, in house MRI obtained through ethical approval, as described in the following, was used to improve training and to test the accuracy on our data.
The US patient data is not ubiquitously available as MRI. Moreover, data compatible with the robotic US scanning system was required. The robot uses a bi-planar US probes that acquires images in the axial and sagittal plane.
To acquire these data, approval of all ethical and experimental procedures and protocols was granted by the ethics committee for clinical trials (CESC) of Verona and Rovigo under application No. 3167CESC and performed in line with the Italian Law Decree No. 158 of 13 September 2012. The ethical approval was the same for MRI and US data.
A number of 293 patients undergoing freehand US-guided trans-perineal prostate biopsies for prostate cancer suspicion at hospital centers in norther Italy were enrolled from April 2021 to July 2022; 242 underwent manual fusion biopsies, with 153 of them having in-house mpMRI Digital Imaging and Communications in Medicine (DICOM) images. A Hitachi Arietta V70 US machine with biplanar transrectal probe was employed. Before performing the biopsy, prostate sagittal and axial US (framerate 20 Hz) were recorded in DICOM format, gathering around 400 images for each patient.
Expert radiologists segmented manually the prostates on sagittal and axial scans with the software of a graphical interface implemented in MeVisLab 2.4 (MeVis Medical Solutions AG, Germany) to create the “ground-truth” for training the AI algorithm. The US dataset was divided into test, validation and training with a ratio of 20%, 20%, and 60%, respectively.
Another cohort of patients for validation was enrolled from February 2023 to June 2023. Transrectal biplanar US acquired with an Esaote Mylab Omega (Esaote SpA, Genoa, Italy) machine was performed, using the same scan protocol.
While the data acquired with the Hitachi Arietta machine was segmented manually, the later data was segmented by integrating the AI algorithm into the MeVisLab 2.4 interface. The interface with AI gave an initial contouring of the prostate and, afterwards, two expert urologists adjusted the AI results, adding more data to the “ground-truth”.
An automatic fusion algorithm that uses the segmented prostate in MRI and US, as well as the original grayscale images was designed. The algorithm works in 3 phases: Pre-alignment, rigid-alignment and elastic fusion. The pre-alignment uses the segmentation centroid and principal axis of the 3D model of the prostate to perform an initial registration (Figure 2). The rigid-alignment phase is based on a registration algorithm that uses the mutual information as a distance measure between MRI and US images and calculates the translation and rotation that minimize the mutual information. After the rigid registration, the contours of the images in MRI and US are used to perform the elastic registration. The elastic registration employs B-Spline function to map one segmentation on the other.
Figure 2. Fusion results. Top row. Left: US and MRI before the registration. The US segmentation is in blue, while the MRI segmentation is red. Middle: US image and segmentations after the pre-alignment. Right: Rigid alignment. Bottom row. Left: Elastic fusion. Middle and Right: Results of the deformable transformation applied to the MRI image. In blue is the contour of the US image, in green the tumour contour in the US image. US, ultrasound.
The goodness of the fusion depends on the accuracy of the segmentation. On the US test data, the Dice coefficient of PROST-Net in prostate segmentation was 0.78. After adding more data with the second cohort, its accuracy increased to more than 0.80. The accuracy of the prostate segmentation in MRI images was computed on the in-house MRIs and the value expressed as Dice coefficient was higher than 0.95.
Dice coefficient between MRI and US segmentation after rigid fusion was 0.75. After fusion, distances between each lesion in MRI to the same lesion in US were assessed.
The mean distance between the lesion detected in MRI and the same lesion detected in US was 8 mm after the rigid fusion and 4 mm after the elastic fusion.
AI FOR AUTOMATIC TUMOUR IDENTIFICATION
AI can play a crucial role in identifying prostate tumors in mp MRIs through various techniques and approaches. Here’s how AI can help in this context.
(1) Image segmentation: AI algorithms can segment the prostate gland and its surrounding tissues in MRI, allowing for a more focused analysis of the relevant regions.[8] Segmentation is a critical step in isolating potential tumor areas.[9]
(2) Machine Learning Models: AI techniques like machine learning and deep learning can be trained on labeled datasets of mpMRIs to recognize patterns associated with prostate tumors. These models can learn to differentiate between malignant and benign tissues based on the extracted features.[10,11]
(3) Fusion of Multiple Parameters: Mp-MRIs provide data from different imaging sequences, including T2-weighted, diffusion-weighted, and dynamic contrast-enhanced images. AI can integrate information from these multiple parameters to improve the accuracy of tumor detection and localization.[11]
(4) Real-time Image Registration: AI algorithms can perform real-time image registration to align MRI sequences with other type of imaging (e.g. US, positron emission tomography [PET]-CT). This ensures that the tumor location is consistent across various images and improves the overall assessment in an image guided biopsy.[12]
In summary, AI’s ability to analyze different images of the prostate can greatly improve the accuracy and efficiency of tumor identification and of an automatic biopsy with a robotic system. It assists in segmenting images, extracting relevant features, and training machine learning models to distinguish between normal and tumor tissue. AI technologies hold the promise of enhancing the diagnostic process and ultimately improving patient outcomes in the detection and treatment of prostate tumors.
COMPARATIVE ANALYSIS AND BENEFITS TO THE PATIENTS
Previously, there have been efforts to enhance the precision of prostate biopsy procedures through the introduction of mechanical systems, software for image fusion and tracking systems. Notable examples include Artemis (innoMedicus AG, Cham, Switzerland) and iSR’obot™ Mona Lisa (Biobot Surgical Pte Ltd, Singapore), both employing a robotic positioner for biopsy needle guidance. MedCom BiopSee® (Medcom GmnH, Darmstadt, Germany) employs a grid system to direct the needle towards predefined target points and a dedicated software for image fusion. Koelis Trinity (Koelis, Meylan, Auvergne-Rhône-Alpes, France) utilizes 3D echography and real-time tracking of the biopsy needle to enhance insertion accuracy. UC Care Navigo (UC-Care Medical Systems, Yoqneam, Israel) monitors the needle’s position during manual insertion and projects it onto prostate images. An alternative strategy to enhance accuracy involves employing a different imaging modality during biopsy. Xact robot (XACT Robotics, Caesarea, Israel) and iSYS1 (iSYS Medizintechnik GmbH, Kitzbühel, Austria) substitute echography with X-rays, presenting a needle guidance system compatible with CT scanners, primarily for abdominal procedures. Soteria Medical (Soteria Medical BV, Arnhem, Netherlands) proposes a system for positioning the biopsy needle during MRI diagnosis. However, both desk research and urologist interviews indicate that these advanced biopsy systems do not significantly outperform standard manual procedures in terms of clinical results, as they lack adequate AI and robotics support for target identification and manual actions.
Indeed, the primary challenge during percutaneous procedures lies in overcoming various challenges such as soft tissue, flexible instruments, awake and breathing patients, among others. Existing navigation and robotic-assisted systems address some of these challenges, but a comprehensive solution is lacking. Therefore, a genuinely hands-free robotic system that mitigates these variables could empower more physicians to treat patients earlier, with heightened accuracy and procedural efficiency. This not only enhances healthcare efficacy but also optimizes resource utilization across different settings (inpatient, outpatient, and ambulatory centers) and facilitates the involvement of mid-level professionals in certain procedures.
The PROST system is a paradigm shift in biopsy procedures that can turn a subjective and operator-dependent procedure into a safer and more accurate diagnostic tool. It allows to treat patients earlier, and its “one–insertion to target” capability makes the procedure predictable and efficient. It requires minimal learning curve, and it is a truly mobile system.
The expected benefits for doctors, patients, and healthcare services
Reduced error in cancer diagnosis
PROST aims to minimize errors in cancer diagnosis by reducing human errors in target selection and biopsy needle aiming. This enhancement in accuracy is expected to achieve a 90% specificity, detecting true negatives, and a 95% sensitivity (dependent on image modalities), surpassing the performance of alternative methods.
Accurate monitoring of disease progression
The system allows for precise monitoring of the disease course. Whether opting for treatment through brachytherapy/ablation or vigilant observation of small lesions, consistent sampling of the exact location at different times provides clinicians with more comprehensive evidence. This assists in deciding the most effective therapy for the patient.
Enhanced treatment processes
A more accurate diagnosis facilitated by PROST contributes to the reduction of false positives and unnecessary prostatectomies. Simultaneously, efforts are made to decrease false negatives, enabling the identification of appropriate surgical interventions and supporting alternative prostate treatments like brachytherapy or ablation.
Ambulatory utilization
Unlike current transperineal biopsy procedures that necessitate a surgical room due to numerous needle entry points, PROST streamlines the process. With a minimal number of entry points (1 to 3, as opposed to up to 24), urologists can conduct biopsies directly in an ambulatory setting, enhancing flexibility and efficiency in healthcare delivery.
CONCLUSIONS AND DIRECTIONS
This paper highlights the significant potential of robotic-assisted prostate biopsy procedures through the inclusion of AI techniques. The integration of AI into the robotic procedure offers multiple advantages, including enhanced accuracy, precision, patient comfort, early cancer detection, and treatment. The key findings and takeaways from this article are as follows.
(1) AI-enhanced prostate biopsy procedures can significantly improve accuracy and precision, leading to more effective patient care and early cancer diagnosis.
(2) AI algorithms play a crucial role in various aspects of the biopsy process, such as target localization, image fusion, needle trajectory planning, and real-time feedback, contributing to the overall success of the procedure.
(3) The use of AI-driven image processing enables the automation of image fusion and the identification and localization of prostate lesions, making the procedure more efficient and reliable.
(4) The combination of robotic assistance and AI helps create an intra-operative model of the prostate, allowing for precise biopsy needle placement and reducing the likelihood of complications.
(5) Real-time AI feedback continuously monitors and adjusts the procedure, minimizing deviations from the planned trajectory and enhancing patient safety.
(6) Ethical considerations surrounding AI integration in robotic prostate biopsy, including data privacy, transparency, and the importance of maintaining clinician expertise, are acknowledged and discussed.
(7) Ongoing research and development in this field are highlighted, indicating a promising future for AI-driven prostate biopsy procedures.
While our primary emphasis has been on advancing prostate biopsy procedures, the AI-based robot-assisted technology we have developed holds substantial promise for broader applications within the realm of prostate-related interventions. The logical progression includes extending its capabilities to encompass low-dose brachytherapy, tumor ablation, and gold marker implantation. This expansion not only enhances the versatility of the technology but also opens avenues for a more comprehensive and nuanced approach to prostate-related medical interventions.
Furthermore, our ongoing efforts involve a thorough exploration of the prerequisites, as well as an assessment of the technical and economic feasibility, to extend this innovative approach to other biopsy procedures. Organs such as the liver, pancreas, and kidneys are under consideration, and we are committed to evaluating the applicability and effectiveness of our technology in these diverse medical contexts.
Importantly, as we extend our focus beyond prostate-related applications, our vision includes achieving a balanced and inclusive application across genders. By ensuring the technology’s suitability for both males and females, we aim to establish a standardized and equitable approach to various medical interventions, thereby contributing to advancements in healthcare across diverse patient populations.
The future of AI and robotics in prostate biopsy looks bright, with the potential to improve patient outcomes and contribute to the field’s advancement.
DECLARATION
Author contributions
The author confirms sole responsibility for the following: Study conception and design, data collection, analysis and interpretation of results and figures, and manuscript preparation.
Source of funding
This research received no external funding.
Informed consent
All the participants in the study provided written informed consent after receiving a detailed explanation of the research objectives, procedures, potential risks, and benefits. The consent process emphasized the voluntary nature of participation, assuring participants that they could withdraw from the study at any point without consequences. All the participants enrolled in this study were adults aged 18 years or older.
Ethics approval
All procedures performed in this study were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards (ethical approval number 3167CESC).
Conflict of interest
Bogdan Maris is an editorial board member of the journal. The article was subject to the journal’s standard procedures, with peer review handled independently of the editor and the affiliated research groups.
Data availability statement
No additional data.
REFERENCES
- Ahmed HU, El-Shater Bosaily A, Brown LC, et al. Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. Lancet. 2017;389(10071):815–822.
- Maris B, Fiazza MC, De Piccoli M, et al. Preclinical validation of a semi-autonomous robot for transperineal prostate biopsy. IEEE Trans Med Robot Bionics. 2022;4(2):311–322.
- Maris B, Tenga C, Vicario R, et al. Toward autonomous robotic prostate biopsy: a pilot study. Int J Comput Assist Radiol Surg. 2021;16(8):1393–1401.
- Yang GZ, Cambias J, Cleary K, et al. Medical robotics-Regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci Robot. 2017;2(4):eaam8638.
- Haidegger T. Autonomy for surgical robots: Concepts and paradigms. IEEE Trans Med Robot Bionics. 2019;1(2):65–76.
- Palladino L, Maris B, Fiorini P. 3d slicer module for semantic segmentation of ultrasound images in prostate biopsy using deep learning techniques. CARS Computer Assisted Radiology and Surgery. 34th International Congress and Exhibition, 2020.
- Palladino L, Maris B, Antonelli A, Fiorini P. Autonomy in robotic prostate biopsy through AI-assisted fusion. In: 2021 20th International Conference on Advanced Robotics (ICAR). IEEE; 2021: 142–147.
- Zavala-Romero O, Breto AL, Xu IR, et al. Segmentation of prostate and prostate zones using deep learning : A multi-MRI vendor analysis. Strahlenther Onkol. 2020;196(10):932–942.
- Pellicer-Valero OJ, Marenco Jiménez JL, Gonzalez-Perez V, et al. Deep learning for fully automatic detection, segmentation, and Gleason grade estimation of prostate cancer in multiparametric magnetic resonance images. Sci Rep. 2022;12(1):2975.
- Wei L, Osman S, Hatt M, El Naqa I. Machine learning for radiomics-based multimodality and multiparametric modeling. Q J Nucl Med Mol Imaging. 2019;63(4):323–338.
- Schelb P, Kohl S, Radtke JP, et al. Classification of Cancer at Prostate MRI: Deep Learning versus Clinical PI-RADS Assessment. Radiology. 2019;293(3):607–617.
- Palladino L, Maris B, Antonelli A, Fiorini P. PROST-Net: A deep learning approach to support real-time fusion in prostate biopsy. IEEE Trans Med Robot Bionics. 2022;4(2):323–326.
