MeshArt: Creating Dynamic 3D Meshes Using Structure-Oriented Transformers
Forget about old, mushy 3D generators—there are now new approaches.
The authors do not attempt to generate a complete, full model all at once. Instead, they first break the object into large blocks or bounding boxes, then analyze how these parts are interconnected. Active use of transformers helps to understand the internal hierarchy and structure of the model. Only at the final stage do they refine the mesh based on its topology.
The MeshArt approach is based on utilizing hierarchical transformers to generate unified 3D meshes with clean and compact geometry that closely resemble manually created models. The creation process occurs in stages and is divided into two major steps.
The first step involves generating the architectural basis of the object, considering how individual parts are connected. Then, based on this structural information, the edges of each detail are modeled. The key aspect of this approach is to model the connections and shapes of elements as a sequence of quantized vectors representing triangles, enabling the formation of a unified hierarchy. Two transformers participate in this process: the first establishes the overall arrangement of parts through their boundary primitives and connection modes, while the second generates the triangular meshes for each part based on these parameters.
Ultimately, this approach is likely better suited for scenes like framed objects, interiors, or exteriors, where the goal is to produce high-quality, precise meshes.
The work is scheduled for release at CVPR 2025 and already garners attention.
More information and a sample video can be found via the link.
The project code is available here.
This system represents an exciting advancement in automatic generation of complex 3D shapes, combining hierarchical concepts and transformers to achieve more accurate and structured models.