This paper describes an integrated system for dental implantation including both preoperative planning utilizing computer-aided technology and automatic robot operation during the intra-operative stage. A novel two-step registration procedure was applied for transforming the preoperative plan to the operation of the robot, with the help of a Coordinate Measurement Machine (CMM). Experiments with a patient-specific phantom were carried out to evaluate the registration error for both position and orientation. After adopting several improvements, registration accuracy of the system was significantly improved. Sub-millimeter accuracy with the Target Registration Errors (TREs) of 0.38±0.16 mm (N=5) was achieved. The target orientation errors after registration and after phantom drilling were 0.92±0.16 ° (N=5) and 1.99±1.27 ° (N=14), respectively. These results permit the ultimate goal of an automated robotic system for dental implantation.

An Integrated Computer-Aided Robotic System for Dental Implantation

Today, OR planning is supported by different clinical information systems. However, relevant patient information is distributed and spread over separate heterogeneous information systems (HIS, RIS, PACS), where each requires its own login and user interaction scheme. Furthermore, transfer of different planning data objects into the OR is currently not accomplished in a consistent electronic manner. This work proposes a framework, which integrates heterogeneous clinical information systems using a portal application with single-sign on and single-patient-lookup functionality. OR planning data can be composed into a digital planning record, which is seamlessly transferred to the modular integrated OR system. Data acquired or generated during surgery is automatically labeled with the electronic patient context, thus ensuring simple and consistent OR documentation.

A framework for data and context integration in the modular digital OR

CustusX version 3 is an in-house application for research in image-guided therapy with a focus on intraoperative use and ultrasound imaging. It is built upon open source toolkits such as the Insight Toolkit (ITK), the Visualization Toolkit (VTK), Qt and others. CustusX has a layered architecture based on services, and is extensible through the use of plugins. Key features are a user interface configurable by the user, an abstraction of the VTK pipeline, cross-platform video acquisition including temporal calibration, and a patient model. This paper gives an overview of the architecture and a description of the key concepts.

CustusX: A Research Application for Image-Guided Therapy

The ARAMIS research platform is a telesurgical robotic system for minimally invasive surgery with focus on autonomous functionality. The original software architecture was a hierarchical one, based on the model-view-controller paradigm. To handle the growing number of devices, we introduce a new framework that facilitates the decomposition of system functionality into separate programs, as well as the data sharing between them. This allows us to integrate and test new functionality more quickly. Our work heavily utilizes the cisst libraries, developed by the Johns Hopkins University. In particular, we take advantage of cisst's multiprocess networking capabilities. As a proof of concept, we demonstrate the integration of an eye-tracking based endoscope control.

A Distributed Software Framework for Robotic Surgery

We have integrated the Philips Research robot arm with the Johns Hopkins cisst library, an open-source platform for computer assisted surgical intervention, for assistance during minimally invasive cardiac surgery using image-based steering of an endoscope. The development of a Matlab to C++ wrapper to abstract away servo-level details facilitates the rapid development of a component-based framework with “plug and play” features. This allows the user to easily exchange the robot with an alternative manipulator while maintaining the same overall functionality.

Integration of a Robotic Endoscope Holder with the Surgical Assistant Workstation Software Framework

Using intensity histograms to estimate image thresholds is a long established practice in image processing and image analysis and a wide variety of techniques have been developed. Different techniques are appropriate for different intensity distributions. This article implements a number of standard techniques not currently available in
ITK.


Histogram-based thresholding - some missing methods

Although many variations of active contour segmentation algorithms exist, most are based on solely edge criteria and breakdown or leak at weak boundaries. One solution to this problem is constraining the segmented area to only statistically possible shapes with the guidance of a shape model. The purpose of this document is to fill the void in the ITK user guide on building active shape models. We describe how to create a 2d active shape model of articular femoral knee cartilage using ITK’s ImagePCAShapeModelEstimator. Sample code and example images are provided for displaying the initial principle components of variation. Shape models built with our code can be used for segmentation with itk::GeodesicActiveContourShapePriorLevelSetImageFilter.

Creating a 2D Active Shape Model Using itk::ImagePCAShapeModelEstimator

This paper presents the rationale for the use of a component-based architecture for computer-assisted intervention (CAI) systems, including the ability to reuse components and to easily develop distributed systems. We introduce three additional capabilities, however, that we believe are especially important for research and development of CAI systems. The first is the ability to deploy components among different processes (as conventionally done) or within the same process (for optimal real-time performance), without requiring source-level modifications to the component. This is particularly relevant for real-time video processing, where the use of multiple processes could cause perceptible delays in the video stream. The second key feature is the ability to dynamically reconfigure the system. In a system composed of multiple processes on multiple computers, this allows one process to be restarted (e.g., after correcting a problem) and reconnected to the rest of the system, which is more convenient than restarting the entire distributed application and enables better fault recovery. The third key feature is the availability of run-time tools for data collection, interactive control, and introspection, and offline tools for data analysis and playback. The above features are provided by the open-source cisst software package, which forms the basis for the Surgical Assistant Workstation (SAW) framework. A complex computer-assisted intervention system for retinal microsurgery is presented as an example that relies on these features. This system integrates robotics, stereo microscopy, force sensing, and optical coherence tomography (OCT) imaging to transcend the current limitations of vitreoretinal surgery.


Component-based software for dynamic configuration and control of computer assisted intervention systems

This paper presents the design of a tele-robotic microsurgical platform designed for development of cooperative and tele-operative control schemes, sensor based smart instruments, user interfaces and new surgical techniques with eye surgery as the driving application. The system is built using the distributed component-based cisst libraries and the Surgical Assistant Workstation framework. It includes a cooperatively controlled EyeRobot2, a da Vinci Master manipulator, and a remote stereo visualization system. We use constrained optimization based virtual fixture control to provide Virtual Remote-Center-of-Motion (vRCM) and haptic feedback. Such system can be used in a hybrid setup, combining local cooperative control with remote tele-operation, where an experienced surgeon can provide hand-over-hand tutoring to a novice user. In another scheme, the system can provide haptic feedback based on virtual fixtures constructed from real-time force and proximity sensor information.

Prototyping a Hybrid Cooperative and Tele-robotic Surgical System for Retinal Microsurgery

Surgery simulation is a technology based on interactive biomechanics as well as haptic and visual rendering for training novice surgeons. Another important component of a surgery simulator is the representation of a patient's anatomy in terms of simple shapes, typically triangles and tetrahedra, coinciding with relevant tissues, whose manipulation, cutting and resection is simulated. While this technology has so far emphasized generic simulators, increasingly researchers are emphasizing models that derive from the images of patients whose pathology collectively are predictive of a significant proportion of the future caseload of surgeons. In addition, our philosophy of producing patient-specific neurosurgery simulators is to organize the architecture in terms of two parameters: the choice of approach- pterional, trans-nasal, frontal and so on, and the nature of the pathology, which in turn determines the specific nature of the treatment and choice of tools. This paper describes on-going work on the refinement of an open-source software pipeline for producing patient-specific neurosurgery simulation, based on segmentation tools such as those available in Slicer and BrainVisa, surface and volume meshing, such as public VTK-based tools and Tetgen respectively, and the SOFA simulation platform. Practical requirements of the various components of the pipeline, such as resolution control and fidelity of surface and volume meshing, and interactive nonlinear mechanics in the biomechanics engine, are discussed.