These rendered 3D scenes with Instant NeRF use the data Hugues previously captured for traditional photogrammetry relying on mirrorless digital cameras, smartphones, 360-degree cameras, and drones. Instant NeRF gives him a powerful tool to help preserve and share cultural artifacts through online libraries, museums, virtual-reality experiences, and heritage-conservation projects. This NeRF was trained using a dataset made of photos taken with an iPhone at the Royal Ontario Museum.
Instant NeRF Artists
Digital artists are composing beautiful scenes and telling stories from a new perspective with NVIDIA Instant NeRF, an inverse rendering tool that turns a set of static 2D images into an immersive 3D scene in minutes.
The Experience
Zeus by Hugues Bruyère
The Last Flowers of Summer by Hugues Bruyère
This impromptu capture of the last days of summer in Montreal provides a timeless memory and free movement around the scene. This render is an example of the power of Instant NeRF to capture and reconstruct the quality of the light of a moment, highlight the specularity of the objects in the scene, and synthesize novel views—all using only a few photos! NeRF (neural radiance fields) opens new possibilities to be explored in real time that are, what Hugues calls, “magical.”
Se Balancer by Franc Lucent
The soft, early-morning sunlight started to peek out through the portes-fenêtres. Walking through the grounds of the château, Franc paused for a brief respite breathing in the fresh air, watching the sun rise into the sky. NeRFs allow the viewer to experience a moment in their own unique way. To not be held by the composition created, but to explore the angles that are most personal and to discover additional perspectives.
Meditation by Franc Lucent
As soon as they walked into one of many rooms in Nico Santucci's estate, Franc knew it needed to be a NeRF. Playing with the dynamic range and reflections in the pond, it presented them with an unknown exploration. They were both really pleased with the softness of the light and how the NeRF elevates the room into something out of a dream—a perfect place to meditate. NeRF is a way to freeze a moment in time that’s more immersive than a photograph or video.
Amalfi Coast by Jonathan Stephens
Jonathan fell in love with the Italian coastline when he backpacked Europe 10 years ago. But the photos he took on the trip couldn’t capture the soul of the towns that dot the Amalfi coastline—their identity is inexplicably linked with their relationship to the mountains and water. NeRFs provide a feeling of space and movement to a scene. The crashing of the waves and the depth of the mountains gives the viewer a greater sense of presence in this magical spot on the planet.
Singapore by Jonathan Stephens
Sunset, reflections, moving clouds—all recipes for a beautiful aerial video. However, the purpose of this piece of art is to unexpectedly transport the viewer into the scene and into a new reality. What appears to be a 2D video quickly becomes an immersive fly-through, giving the viewer a much greater appreciation of scale and space. NeRFs unlock unlimited views and unshackle creators from the confines of the original shot. Layers of light become a playground to move through and push the imagination to new worlds.
Through the Looking Glass by Karen X. Cheng and James Perlman
A pianist practices her song, part of her daily routine. Though there’s nothing average or routine about what happens next. The viewer peers into the mirror, a virtual world that can be observed but not traversed; it’s unreachable by normal means. Then, crossing the threshold, it’s revealed that this mirror is in fact a window into an inverted reality that can be explored from within. Which one is real?
NeRF allows us to easily build photorealistic scenes with surreal features, and that’s truly beautiful. This piece was inspired by a previous work originally created by Hugues Bruyère, aka @smallfly.
The Process
Capturing imagery is the first step to getting a beautiful Instant NeRF. Take photos from many different perspectives following basic principles of photogrammetry with respect to overlapping and sharp imagery. This video shows you example imagery from an ideal capture.
Download and compile the codebase, then launch the GUI to train your first NeRF. The codebase is straightforward to compile for experienced programmers and data scientists. Beginners can easily follow these tips to get started, and the instructions provided in GitHub repository.
Using spatial location and volumetric rendering, the Instant NGP model uses the camera pose from the images to render the 3D space of the scene. The longer the AI trains on the images, the more details are rendered. New camera paths can be created for numerous renders and explorations of the new 3D scene.
About the Artists
About Hugues Bruyere
Hugues Bruyere is the chief of innovation and co-founder of Dpt, an award-winning studio that meshes the virtual and physical. For the past 20 years, Hugues has embraced new technologies and creative explorations as a means to create interactive and immersive experiences.
About Karen X Cheng
Karen X Cheng is a director known for sharing her creative experiments on Instagram (@karenxcheng 1.2M). She works closely with tech companies on their latest products, finding innovative and creative uses to inspire creators. She has been named in 30 under 30 and Adweek's Creative 100 for her work.
About James Perlman
James Perlman is an artist who uses his experience building software to turn dreams into reality. He frequently uses cutting-edge research in his creative process. Lately, he has been building on top of Instant-NGP to push the limits of what’s possible with machine-accelerated imagination.
About Jonathan Stephens
Jonathan Stephens is the chief evangelist and marketing director at EveryPoint. Beginning his career as a GIS analyst, Jonathan pivoted into computer vision and image processing technology as a solutions consultant focusing on solving inventory management and logistics problems for bulk material producers. In his current role at EveryPoint, Jonathan’s main focuses are in market discovery, research, and growing customer awareness through technical education.
Featured Sessions
NVIDIA Instant NeRF: Getting Started with Neural Radiance Fields
Hear from the researchers to learn more about what Instant NeRF can do to help with your 2D-to-3D visualization development projects.
In as little as an hour, you can compile the codebase, prepare your images, and train your first NeRF. Unlike other NeRF implementations, Instant NeRF only takes a few minutes to train a great-looking visual.
Instant Neural Graphics Primitives
We'll show how neural graphics primitives can be trained in seconds rather than hours, as well as rendered at faster-than-real-time rates (>60fps >1080p) without any post-processing of the trained model. We'll focus on three key ingredients:
• A new neural input encoding
• a new NVIDIA® CUDA® training and inference framework
• and efficient algorithms for differentiable rendering of the graphics primitives
Introducing NVIDIA Kaolin Wisp: A Common Library for Neural Field Research
NVIDIA Research teams are introducing Kaolin Wisp, a research-oriented library for neural fields, providing a common library and framework for neural field research.
Watch this session to learn how researchers, developers, or even creators can leverage this new open library to accelerate and simplify neural field workflows.