NVIDIA CUDA: Naked-Eye Stereoscopic System for Real-Time Medical Imaging
One interesting field of imaging technology is naked-eye stereoscopy, which displays 3D stereoscopic images without the need for special eyeglasses. This intriguing technology not only has applications in entertainment, but is being studied as a practical technology for a variety of professional applications. One especially promising application is in medical imaging, where NVIDIA’s CUDA™ parallel computing platform is being studied by Professor Takeyoshi Dohi and his colleagues in the Department of Mechano-Informatics at the University of Tokyo’s Graduate School of Information Science and Technology.
| Principle of Integral Videography (IV) |
Naked-eye stereoscopy can be implemented in a variety of ways; the one being studied by Associate Professor Hongen Liao and graduate student Nicholas Herlambang in Profesor Dohi’s research group is called Integral Videography (IV). This method uses a special display comprising a micro-lens array, consisting of convex lenses on a matrix which is bonded to a liquid crystal panel. Directly beneath each micro-lens, there are some 100 liquid crystal elements and the convex lens projects the light from each element in various directions. The object to be represented in 3D space is illuminated by light rays from several directions, forming a stereoscopic image which to the user seems to be floating in the air.
Because this method projects a 3D image into space, it has advantages over the traditional stereoscopic method, where different images are displayed for the viewer’s left and right eyes. Using IV, the 3D image can be observed from a wide area in front of the display by several viewers at once, without using special eyeglasses or viewpoint tracking.
Since 2000, the University’s research group has been developing a system where in vivo cross-sections obtained in real time by CT or MRI scans are treated as volume textures, which can not only be reconstituted as 3D images through volume rendering, but also displayed as stereoscopic video for use in an IV system.
This system could revolutionize real-time, stereoscopic, in vivo imaging. However, the amount of computation is huge; the volume rendering alone creates a high processing load, then further processing is required for the stereoscopic imaging. For each video frame, a vast number of angles must be displayed at the same time. Multiply this by the number of frames in the video and a staggering amount of computation must be done with high precision in a short time.
In research in 2001, real-time volume rendering and stereoscopic reconstitution for images of 512 x 512 resolution on a Pentium III 800 MHz PC took over ten seconds to generate a single frame. To speed up processing, the group tried using the 60 CPUs of an UltraSPARC III 900 MHz machine, the latest high-performance computer available at the time. But the best result that could be obtained was five frames per second. This was simply not fast enough to be practical.
|Example of IV images for viewing from a distance. A highly realistic, stereoscopic image of the yellow rod is formed on a hand two meters in front of the display. Even when the observer moves, the image remains visible on his hand. To create a high-resolution, stereoscopic image, processing methods such as volume rendering are used, but they require enormous computing capability. |
Both the volume rendering and subsequent conversion to IV format require data-parallel vector calculation. For this, the optimal computing paradigm is the GPU. Accordingly, Liao and Herlambang started to research GPU implementation using CUDA, a general-purpose, C-language GPU development environment from NVIDIA.
First, the researchers developed a prototype system using the GeForce® 8800 GTX, a latest-generation GPU. When data sets from the 2001 study were run on the GPU using CUDA, performance improved to 13-14 frames per second. As the UltraSPARC system had cost tens of millions of yen, the researchers were amazed that a GPU, costing a hundred times less, delivered nearly three times the performance. Moreover, according to the group, NVIDIA’s GPU was at least 70 times faster than the latest generation of multi-core CPUs. In addition, tests showed that the GPU’s high performance was even more conspicuous for larger volume texture sizes.
Currently, the group is working with NVIDIA’s latest deskside supercomputer, the Tesla™ D870 and is optimizing the current IV system for Tesla using CUDA. This is expected to boost performance even further.
|IV system using CUDA |
“We have evaluated various development environments for parallel computing,” says Herlambang, the researcher responsible for the CUDA implemention, “but we chose CUDA because it lets us do development in the C language syntax we are used to. Also, we will be able to take advantage of speed increases in future generations of GPUs without modifying the system we have developed. If an environment that makes it easy to debug large CUDA programs becomes available, CUDA will become an even more powerful development environment for parallel computing and we expect it will find more applications in medical image processing as well.”
When images from CT and MRI are viewed stereoscopically in real time, physicians can check the state of diseased tissues and make diagnoses without biopsies and surgery. Moreover, several physicians can view the images at the same time and consult with one another. And it may eventually make it possible for several physicians to perform arthroscopic surgery and other minimally invasive surgical techniques together, with each surgeon able to visualize the operation in real time.
It is difficult to bring a huge parallel computer array into clinical settings, but the powerful computing capabilities of GPUs and Tesla make it possible to provide compact, parallel computing modules.
Prof. Takeyoshi Dohi, Advanced Therapeutic & Rehabilitation Engineering Laboratory: //www.atre.t.u-tokyo.ac.jp/en/
Associate Professor Hongen Liao: //bmpe.t.u-tokyo.ac.jp/~liao/
▼Introduction to Laboratory
ATRE Lab, Dept. of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, where Prof. Dohi and his colleagues have pioneered computer-aided surgery, is one of the world’s top laboratories in the medical engineering field.
The 3D stereoscopy technology presented here has given rise to numerous advanced devices, including microsurgery robots, wedge prism endoscopes, and surgical stereoscopic composite displays.