This paper presents LEGO®-Maker, a new learning-based generative model that can effectively consider over 100 unique brick types and rapidly generate hundreds of bricks to create LEGO® models conditioned on images. This work has three major technical contributions that enable it to achieve surpassing capabilities beyond existing generative approaches. First, we design a compact LEGO® tokenization scheme to serialize LEGO® models and bricks into tokens for autoregressive learning. Second, we build LEGO®-Maker, an autoregressive image-conditioned architecture, with a multi-token prediction strategy to encourage pre-considering multiple brick attributes and a rollback mechanism for collision-free generation. Third, we propose an effective data preparation pipeline with a procedural generator to synthesize LEGO® models and a LEGO®-to-real image translator distilled from a large vision language model to translate LEGO® renderings into associated photorealistic images, leveraging rich prior to address the scarcity of image-to-LEGO® data. Extensive evaluations and comparisons are conducted on two object categories, facade and portrait, over metrics in four aspects: geometry, color, semantics, and structural integrity, together with a user study. Experimental results demonstrate the versatility and compelling strengths of LEGO®-Maker in producing structures and details given by the reference image. Also, the evaluation scores manifest that our method clearly surpasses the baselines, consistently for all evaluation metrics.

LEGO®, autoregression, machine learning, 3D generation, assembly generation

J. Ge, M. Zhou, H. Zheng, H. Xu, C.-W. Fu “LEGO®-Maker: Autoregressive Image-Conditioned LEGO® Model Creation,” ACM Transactions on Graphics (TOG), vol. 44, 1–15, 2025.

@article{ge2025lego,
title={LEGO®-Maker: Autoregressive Image-Conditioned LEGO{\textregistered} Model Creation},
author={Ge, Jiahao and Zhou, Mingjun and Zheng, Hanyou and Xu, Hao and Fu, Chi-Wing},
journal={ACM Transactions on Graphics (TOG)},
volume={44},
number={6},
pages={1--15},
year={2025},
publisher={ACM New York, NY, USA}
}