GET3D is an innovative technology developed by NVIDIA that utilizes a generative model to create high-quality 3D textured shapes from images. This technology, which was announced at NeurIPS 2022, uses machine learning techniques to generate textured meshes that can be directly consumed by 3D rendering engines, and thus can be used immediately in downstream applications.
These 3D models are generated using generative adversarial networks (GANs), a class of machine learning frameworks. GANs are designed to generate new, synthetic instances of data that can pass for real instances.
The GET3D project is open-source and is implemented in PyTorch. Developers, researchers, and hobbyists who are interested in the capabilities of machine learning for 3D modeling can utilize this tool.
In the context of the ongoing growth of 3D virtual worlds and the metaverse, tools like GET3D are crucial. NVIDIA, for instance, is using generative AI to speed up the creation of virtual worlds and introduce new possibilities for creators across various industries.
GET3D generates 3D shapes with arbitrary topology, high-quality geometry, and texture, making it a powerful tool for industries moving towards modeling massive 3D virtual worlds.
The output from GET3D can be used in a variety of 3D formats, including OBJ, PLY, and STL. This broad compatibility makes GET3D a versatile tool for generating 3D content.
It is notable that prior work on 3D generative modeling often lacked geometric detail, were limited in the mesh topology they could produce, typically didn’t support textures, or used neural renderers in the synthesis process, which made their use in common 3D software non-trivial. GET3D overcomes these limitations and represents a significant step forward in the field of 3D generative modeling.
The GET3D tool can be run on a local machine with a compatible GPU or in the cloud using NVIDIA’s NGC. Detailed documentation on how to use the tool, as well as links to additional resources and tutorials, can be found on the project’s GitHub page.
