A software architecture has been developed for image-guided robot-assisted cell manipulation. The system provides tools for positioning and navigation of multiple micromanipulators under live microscope images. The main application of this system is in multi-electrode patch clamp electrophysiology; however the same architecture can be used for any other type of cell micromanipulation as well as other applications such as micro-assembly.

A software architecture for cell manipulation

As robotic surgery gains popularity [1, 2, 3], methods for improving situational awareness during tele-
operation have become an active area of research. Literature has attempted to incorporate haptic feedback
displays to enhance and improve user performance. For example, Massimino et al. [4] showed that a combi-
nation of vibrotactile and auditory substitutions lead to task performance (peg-in hole task) comparable to
that using a force feedback. Kitagawa et al. [5] extended this approach by using visual force displays and
and auditory cues, in experiments showing comparable performance in surgical tasks (knot-tying). Reiley et
al. [6] used a visual force display in a teleoperated knot-tying task to demonstrate lower forces and reduced
suture breakage by trainees.
The above art demonstrates the need for information overlays in telerobotic surgical tasks. However, this
literature also used prototype software and tools intended only for the specic experiments. By contrast,
we use the Surgical Assistant Workstation (SAW) [7, 8] in development at Johns Hopkins University to
create a general information overlay, and demonstrate its utility by creating a visual warning display for
telerobotic surgery that detects instruments being operated outside of the eld of view of the endoscopic
camera. SAW is a modular framework for rapid prototyping of new tools and methods for robotic surgery. It
includes methods for image guidance, registration with pre-operative and intra-operative images, and ability
to interact with the graphical objects rendered within the display with the master or slave manipulators in
a teleoperation environment.
The common telesurgical system in use is the da Vinci Surgical System (Intuitive Surgical Inc.). It
consists of a surgeon's console containing the two master manipulators, a patient side cart with up to
four robotic arms - three for the slave instrument manipulators which can be equipped with the removable
instruments and an endoscope camera manipulator connected to a high-performance stereo vision system.
The da Vinci also provides a research and development application programming interface (DiMaio, et al.,
[9]) that streams kinematics data and system events at congurable rates of up to 100Hz. The SAW/cisst
framework also contains an interface to the da Vinci API.
We present an overlay architecture (Figure 1) implemented using the cisst/SAW libraries to integrate
contextual procedure and system information for improving safety, and situational awareness during these
delicate and complex manipulations. While the presented methods can be modied for use with any robotic
system, we used our da Vinci S Surgical System (Intuitive Surgical Inc.) for the validation experiments
(Figure 2) here. Results from validation experiments with 17 users and a total of 50 training sessions
totaling 214350 image frames are presented.

Kinematics Based Safety Operation Mechanism for Robotic Surgery extending the JHU SAW Framework

The Image-Guided Surgery Toolkit (IGSTK) provides tracker interfaces for various tracking devices. The tracker component of IGSTK was extended with three new tracking interfaces: CamBar B2, EasyTrack 500, and Active Polaris. Using an IGSTK application we evaluated the precision of each of the tracking systems. Based on our evaluation we conclude that all tracking systems are sufficiently accurate for ENT procedures.

Extending the tracking device support in the Image-Guided Surgery Toolkit (IGSTK): CamBar B2, EasyTrack 500, and Active Polaris

This document presents a VTK wrapper of an extracted portion of ‘3DTK - The 3D Toolkit’
(http://threedtk.de) to enable a developer to ﬁnd planes in 3D point cloud data.

Hough Transform Plane Detector

Purpose
The lack of interfaces for system integration in the Operating Room (OR) has been recognized as one of the major factors to impede efficient OR turnover procedures as well as efficient intraoperative data handling and ergonomics in the OR. A software interface for application-layer connection between data processing units in the OR is presented.
Methods
The requirements for data exchange in the OR were derived, mostly from the literature. A c++ class library was developed according to these requirements. The efficiency of the data exchange and the limits of the system were investigated by a series of experiments. An examplary application was built using the library to show its applicability under realistic conditions.
Results
We present the software library TiCoLi for automatic OR network configuration, device and service discovery ("surgical plug-and-play"), message exchange, data streaming, and remote access to device parameters and functionalities through an Ethernet network. The system is based on a modular, object-oriented software design, which guarantees high flexibility regarding the adaptation of its subsystems to the needs of a particular clinical application.
Conclusions
The TiCoLi package is the first open source software library which was developed for system integration in the OR which covers message exchange, data streaming, and auto configuration.

TiCoLi - An Open Software Library for Device Integration in the Digital OR based on Open Software and Open Standards

In order to test our software GoFigure2 for the
visualization and the processing of extremely large 3D+t microscopy images,
we utilized the visual programming technology Sikuli to
automate our GUI testing. We then integrated these Sikuli tests into CTest and
used CDash to report the results on a dashboard with the view to test our
software automatically every night and detect bugs as soon as they occur. Our
rate of releases more than doubled as a result.

CTest Integration of Sikuli Automated GUI Testing

In this paper, we review the technical details of new features in OpenIGTLink version 2. OpenIGTLink is a free open-source network communication protocol designed for data exchange among devices and software. Since the first release of version 1.0 protocol along with a reference C/C++ library in 2008, the number of developers and users, who use OpenIGTLink in their research projects, have been growing among convincing some commercial companies to implement research interface based on OpenIGTLink their commercial products. However, the message types initially defined for research prototypes does not cover some of functionality in commercial products, thus the protocol needed to be requires refinement in the protocol. The goal of the extension in version 2 is to support such new demands emerged after the release of the initial version, while keeping the protocol as simple as possible. The version 2 protocol has been implemented as an extension of version 1; it is backward compatible in the sense that messages generated by OpenIGTLink version 2 compliant software can be received by version 1 compliant software. 

OpenIGTLink Protocol Version 2

A C++ file reader for the VisualSonics Vevo® 770 high frequency ultrasound system is presented. The digital RF header file is read into an XML DOM representation, which can be serialized as XML, HTML, or accessed in C++ natively with data object binding. An InsightToolkit reader is also implemented.

A File Reader for the VisualSonics Vevo 770 Digital RF Data

This document describes an affine transformation algorithm as an additional feature for landmark based registration in ITK www.itk.org. The algorithm is based on the paper by Spath, H [1]. The author derives a set of linear equations from paired landmarks and generates an affine transform from them. The method implemented here gives more freedom in the choice of registration and/or initialization method in ITK. The submission describes ITK implementation of the algorithm.

Affine Transformation for Landmark Based Registration Initializer in ITK

ABSTRACT
Brain tissue segmentation is important in many medical image applications. We augmented the Expectation-Maximization segmentation algorithm in Slicer3 (www.spl.harvard.edu) . Currently, in the EM Segmenter module in Slicer3 user input is necessary to set tissue-class (Gray Matter, White Matter etc.) intensity values. Our contribution to the current pipeline is to automatically compute such values using k-means clustering. Our method can be applied to scans of varying intensity profiles and thus we obviate the need for a normalization step. We applied this pipeline on multiple datasets and our method was able to accurately classify tissue-classes. The implementation was done as a standalone utility in the Python programming language (www.python.org) and work is underway to incorporate it in the EM processing pipeline. 