NVIDIA Unveils fVDB Framework for Enhanced Digital World Models
Ted Hisokawa Jul 30, 2024 07:40
NVIDIA introduces fVDB, a deep-learning framework for creating AI-ready virtual representations, enhancing applications in autonomous vehicles, climate science, and smart cities.
NVIDIA has announced the launch of fVDB, a new deep-learning framework designed to generate AI-ready virtual representations of the real world. The announcement was made at SIGGRAPH, a prominent conference for computer graphics and interactive techniques, according to the NVIDIA Blog.
Revolutionizing Digital Representations
Built on the foundation of OpenVDB, the industry-standard library for simulating and rendering sparse volumetric data, fVDB is poised to significantly enhance the creation of virtual environments. These environments are crucial for training AI in various applications, including autonomous vehicles, climate science, and smart cities.
OpenVDB has been instrumental in simulating elements like water, fire, smoke, and clouds. By leveraging this technology, fVDB can capture large-scale and fine-detailed aspects of real-world environments, translating them into AI-ready data rendered in real time. This capability is essential for developing generative physical AI, such as robots and autonomous vehicles, which require a deep understanding of 3D space.
Technological Advancements and Applications
The framework builds on a decade of innovation with OpenVDB, introducing enhancements that allow industries to benefit from digital twins of the real world. These virtual environments are used for training autonomous agents, capturing city-scale 3D models for climate science, and planning urban spaces, among other uses.
fVDB incorporates NVIDIA-accelerated AI operators on top of NanoVDB, a GPU-accelerated data structure for efficient 3D simulations. These operators include convolution, pooling, attention, and meshing, designed for high-performance 3D deep learning applications. This development enables businesses to build complex neural networks for spatial intelligence, including large-scale point cloud reconstruction and 3D generative modeling.
Key Advantages of fVDB
- Larger: 4x larger spatial scale compared to previous frameworks.
- Faster: 3.5x faster processing speed.
- Interoperable: Full utilization of massive real-world datasets, with VDB datasets rendered into full-sized 3D environments in real time.
- More Powerful: 10x more operators than previous frameworks, simplifying complex processes by combining functionalities that previously required multiple deep-learning libraries.
Future Integration and Availability
fVDB will soon be available as part of NVIDIA NIM (NVIDIA Inference Microservices). These microservices will enable businesses to integrate fVDB into OpenUSD workflows, generating AI-ready OpenUSD geometry in NVIDIA Omniverse, a platform for industrial digitalization and generative physical AI applications. The microservices include:
- fVDB Mesh Generation NIM — Generates digital 3D environments of the real world.
- fVDB NeRF-XL NIM — Produces large-scale NeRFs in OpenUSD using Omniverse Cloud APIs.
- fVDB Physics Super-Res NIM — Performs super-resolution to generate high-resolution physics simulations based on OpenUSD.
Continued Innovation in OpenVDB
Over the past decade, OpenVDB has earned multiple Academy Awards as a core technology in the visual-effects industry and has expanded into industrial and scientific applications. Four years ago, NVIDIA introduced NanoVDB, adding GPU support to OpenVDB for faster performance and real-time simulation and rendering. Two years ago, the company launched NeuralVDB, which incorporates machine learning to significantly compress the memory footprint of VDB volumes.
fVDB represents the latest advancement, building AI operators on top of NanoVDB to unlock spatial intelligence at the scale of reality. Interested parties can apply to the early access program for the fVDB PyTorch extension. The framework will also be available through the OpenVDB GitHub repository.
Image source: Shutterstock