Panelformer: Sewing Pattern Reconstruction from 2D Garment Images

Abstract

In this paper, we present a novel approach for reconstructing garment sewing patterns from 2D garment images. Our method addresses the challenge of handling occlusion in 2D images by leveraging the symmetric and correlated nature of garment panels. We introduce a transformer-based deep neural network called Panelformer that learns the parametric space of garment sewing patterns. The network comprises two components: the panel transformer and the stitch predictor. The panel transformer estimates the parametric panel shapes, including the occluded panels, by learning from the visible ones. The stitch predictor determines the stitching information among the predicted panels, enabling the reconstruction of the complete garment. To mitigate the overfitting problem caused by strong panel correlations, we propose two tailor-made data augmentation techniques: panel masking and garment mixing. These techniques generate a wider variety of panel combinations, enhancing the model's robustness and generalization capability. We evaluate the effectiveness of Panelformer using a synthetic dataset with diverse garment types. The experimental results demonstrate that our method outperforms competing baselines and achieves comparable performance to NeuralTailor, which operates on 3D point cloud data. This validates the efficacy of our approach in the context of garment sewing pattern reconstruction. By utilizing 2D images as input, our method expands the potential applications of garment modeling and offers easy accessibility to end users.

Methodology

Architecture overview.The architecture consists of two main components: the panel transformer and the stitch predictor. The input image I is first feed into our panel transformer to reconstruct the shapes and placement of panels. We later use an MLP classifier to predict the stitching information to create a complete sewing pattern.

Data augmentation.We introduce two data augmentation techniques named panel masking and garment mixing. We first acquire the bounding box of each panel leveraging the projected ground truth vertex segmentation of the garment image, then randomly remove or copy-paste the content within the segmentation bounding boxes to introduce variations and increase the diversity of the training dataset. These augmentation techniques effectively prevent overfitting and improve the robustness of the model.