3D goes live with Mediapro and NVIDIA
Images Courtesy of Mediapro
There’s nothing more exciting than live sporting events, with all the sights, sounds and action of game day. With recent 3D television technology breakthroughs, broadcasters can bring games directly into the living room, delivering the immersive experience all avid sports fans crave. Broadcasters are now creating live, compelling, immersive content that keeps fans tuning in for more. Viewers can experience hurling soccer balls or blazingly fast sports cars without the crowds, parking nightmares and high ticket prices.
However live 3D broadcasts present broadcasters with enormous technological challenges. Unlike movies or TV shows, live events do not have the luxury of post-production processes. Broadcasting live 3D requires that each scene be filmed simultaneously by two cameras: one to capture what the left eye sees, the other to capture what the right eye sees. The images from these two cameras must be perfectly aligned, with the exact same lenses, focal length, orientation, and so on. Because there is no time for post production, these cameras must instead rely on complex image processors or image analyzers to synchronize the stereo pairs in real time.
Traditionally, stereo 3D production has required very high-priced technology. However, the research and development division of Mediapro, a group of production and broadcasting companies with headquarters in Barcelona, has designed a practical, affordable real-time image analyzer that enables live events to be shown in 3D, by leveraging NVIDIA's CUDA-enabled Quadro Digital Video Pipeline.
Medialuso, Mediapro’s production company in Portugal and the first user of this NVIDIA-powered system, has already used the technology to broadcast a number of high-profile live 3D events worldwide. Viewers in Europe, Asia and elsewhere with access to 3D TVs or 3D-equipped theaters are exhilarated by Medialuso-produced live soccer games, tennis matches, World Cup surfing contests and concerts in real-time 3D.
"The remarkable difference of watching an event in 3D is that you feel so much closer to the action," says Sergi Sagas, Scientific Director of Mediapro's research department. "You find yourself ducking as soccer balls seem headed right toward you, or raising your hand to your face as if a breaking wave is showering you with water. We are very excited to be part of a movement that is making it practical for more people to experience the wonders of live 3D for themselves.”
Making live 3D broadcasting practical
"3D requires precisely coherent stereo images, which means taking tremendous care to align the pair of cameras shooting every scene," says Sagas. "For live 3D, fixing all the mismatches between the two cameras has to be automatic and instantaneous, and the only way to do it is with intelligent and very fast image processing." This is exactly the basis of Mediapro's image processing system for producing live 3D stereo events.
"Our Quadro-powered systems are about one-tenth the cost of other live 3D processing tools available today," says Sagas. "While other systems use entirely proprietary technology, we have developed a system with flexibility that comes from technology built on standards."
Performance with no margin for error
Mediapro counts on NVIDIA's reliability to make everything work. Live broadcasts are filled with risk and equipment needs to work flawlessly so Mediapro turns to NVIDIA technology to deliver without fail.
"The 3D signal we produce is seen all over the world, and there's tremendous pressure for us to perform perfectly," says Sagas. "Our customers depend on us to deliver what we promise, and we couldn't do it without NVIDIA."
Mediapro's real-time image analyzer reads the two HD video streams from a stereo pair of video cameras, processing the images in real time to compensate for any color variations, alignment or rotation discrepancies, and other transformations between the two cameras' images. The Quadro-based software also controls specially designed robotic camera rigs from its partner Kronomav, another Spanish company. The software signals the Kronomav rigs to alter, in real time, the focal length, color balance, rotational angle, and other parameters required for a camera pair to produce exactly aligned right- and left-eye images.
Each pair of cameras is automatically corrected with Mediapro's real-time image processing system: a powerful workstation, which uses the NVIDIA Quadro Digital Video Pipeline to acquire, correct, and output two HD-SDI video feeds. This Quadro-powered solution leverages the parallel processing power of CUDA to calculate color matching, alignment, and all the other parameters needed to align and correct each pair of cameras – live.
The Quadro Digital Video Pipeline is the industry's only GPU-accelerated solution for real-time acquisition, processing, and delivery of high-resolution video across both standard and 3D video broadcast environments. In addition, it is the only platform to deliver up to four HD-SDI video inputs to each GPU card. while keeping the GPU processing power fully available for processing graphics.
"To produce precisely coherent stereo images in real time, all the processing must happen extremely fast," says Jordi Alonso, a Senior Researcher in Mediapro's research department and a specialist in 3D technologies. "The CUDA-enabled Quadro solution is the only technology we considered seriously. It allows us to process massive amounts of data in real time and with virtually no latency between SDI in and SDI out on the system."
"NVIDIA has worked very collaboratively with us, helping us optimize all the features of the technology and supporting us throughout the development of our 3D real-time image analyzer system," adds Sagas.
Mediapro continues to refine its NVIDIA-based systems, to make it easier for production crews to produce live 3D events. For example, the Mediapro research team is working on enhancements that include even better depth computation – calculating depth maps that indicate the distance of an object from each of a stereo pair of cameras; automatic equalization of disparities in all the production cameras, to help directors switch smoothly between multiple camera pairs shooting an event; and still better interface and communication with hardware rigs, for faster automatic calibrations.
"The Quadro Digital Video Pipeline technologies have allowed us to create real-time 3D production capabilities that others have not yet achieved and at a price no one else can touch, which is keeping Mediapro more competitive globally," says Sagas.