ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction

Abstract

Directly generating 3D meshes, the default representation for 3D shapes in the graphics industry, using auto-regressive (AR) models has become popular these days, thanks to their sharpness, compactness in the generated results, and ability to represent various types of surfaces. However, AR mesh generative models typically construct meshes face by face in lexicographic order, which does not effectively capture the underlying geometry in a manner consistent with human perception.

Inspired by 2D models that progressively refine images, such as the prevailing next-scale prediction AR models, we propose generating meshes auto-regressively in a progressive coarse-to-fine manner. Specifically, we view mesh simplification algorithms, which gradually merge mesh faces to build simpler meshes, as a natural fine-to-coarse process. Therefore, we generalize meshes to simplicial complexes and develop a transformer-based AR model to approximate the reverse process of simplification in the order of level of detail, constructing meshes initially from a single point and gradually adding geometric details through local remeshing, where the topology is not predefined and is alterable.

Our experiments show that this novel progressive mesh generation approach not only provides intuitive control over generation quality and time consumption by early stopping the auto-regressive process but also enables applications such as mesh refinement and editing.

Simple Demo (Generating)

A simple demonstration shows how our method can generate a cube and a tetrahedron together. This video visualizes the smooth geomorphing of level-of-detail approximations.

Object-Level Generation

Our method can be used to directly encode an object or learn from a collection of objects with high fidelity, generating in a coarse-to-fine manner auto-regressively. The results below are generated from the ShapeNet or Thingi10k datasets.

(The slider on the left indicates the levels of detail that our algorithm generates in an auto-regressive manner. Mesh simplification corresponds to a coarsening process that ultimately leads to a single point, and reversing it can be considered a generation process.)

Scene-Level Representation

Our method can also be used to represent a complex indoor scene consisting of multiple objects, delivering results in a streaming manner. Isolated points are visualized as spheres. The results below are from the 3D-FRONT dataset.

(The slider on the left indicates the levels of detail that our algorithm generates in an auto-regressive manner. Mesh simplification corresponds to a coarsening process that ultimately leads to a single point, and reversing it can be considered a generation process.)

Comparison to Concurrent Work

There is concurrent work named VertexRegen that has performed a similar task to ours. The major difference is that they are built upon Progressive Meshes, while we are built upon Progressive Simplicial Complexes. This implies the following:

Dimension. Meshes are special cases of simplicial complexes. Specifically, meshes are only composed of triangles, excluding isolated points or line segments. This indicates that the range of geometric objects our method can represent is a superset of those of their method.
Topology. The above property also suggests that our approach has the ability to handle topological changes (i.e., generating any 3D meshes from the same starting point, regardless of how complex the target object is), whereas their method inherently cannot. This is because their core operation of edge collapse is designed to preserve the local topology, making the overall topology of the generated mesh never change.

BibTeX

@misc{lei2025armesh,
      title={ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction}, 
      author={Jiabao Lei and Kewei Shi and Zhihao Liang and Kui Jia},
      year={2025},
      eprint={2509.20824},
      archivePrefix={arXiv},
      primaryClass={cs.GR},
      url={https://arxiv.org/abs/2509.20824}, 
}