This paper presents a computational pipeline for creating personalized, physical LEGO® figurines from user-input portrait photos. The generated figurine is an assembly of coherently-connected LEGO® bricks detailed with uv-printed decals, capturing prominent features such as hairstyle, clothing style, and garment color, and also intricate details such as logos, text, and patterns. This task is non-trivial, due to the substantial domain gap between unconstrained user photos and the stylistically-consistent LEGO® figurine models. To ensure assemble-ability by LEGO® bricks while capturing prominent features and intricate details, we design a three-stage pipeline: (i) we formulate a CLIP-guided retrieval approach to connect the domains of user photos and LEGO® figurines, then output physically-assemble-able LEGO® figurines with decals excluded; (ii) we then synthesize decals on the figurines via a symmetric U-Nets architecture conditioned on appearance features extracted from user photos; and (iii) we next reproject and uv-print the decals on associated LEGO® bricks for physical model production. We evaluate the effectiveness of our method against eight hundred expert-designed figurines, using a comprehensive set of metrics, which include a novel GPT-4V-based evaluation metric, demonstrating superior performance of our method in visual quality and resemblance to input photos. Also, we show our method's robustness by generating LEGO® figurines from diverse inputs and physically fabricating and assembling several of them.

LEGO®, computational design, fabrication, assembly, appearance adaptation, image synthesis

J. Ge, M. Zhou, W. Bao, H. Xu, C.-W. Fu “Creating LEGO figurines from single images,” ACM Transactions on Graphics (TOG), vol. 43, 1–16, 2024.

@article{ge2024creating,
title={Creating LEGO figurines from single images},
author={Ge, Jiahao and Zhou, Mingjun and Bao, Wenrui and Xu, Hao and Fu, Chi-Wing},
journal={ACM Transactions on Graphics (TOG)},
volume={43},
number={4},
pages={1--16},
year={2024},
publisher={ACM New York, NY, USA}
}