Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines


FengGen Yu, Yan Zhang, Kai Xu, Ali Mahdavi-Amiri, Hao Zhang

Accepted to ACM Transactions on Graphics(2018)


Given a heterogeneous 3D shape collection (a), we perform style co-analysis over projected feature lines (see insets) to spatially located style patches (b) and cluster the shapes based on their styles — all the four shapes in color belong to the same cluster. Spatial localization of style patches enables applications such as style-preserving mesh simplification (c-d). Note the denser triangle distributions near style patches (d).

Abstract


We present a semi-supervised co-analysis method for learning 3D shape styles from projected feature lines, achieving style patch localization with only weak supervision. Given a collection of 3D shapes spanning multiple object categories and styles, we perform style co-analysis over projected feature lines of each 3D shape and then backproject the learned style features onto the 3D shapes. Our core analysis pipeline starts with mid-level patch sampling and pre-selection of candidate style patches. Projective features are then encoded via patch convolution. Multi-view feature integration and style clustering are carried out under the framework of partially shared latent factor (PSLF) learning, a multi-view feature learning scheme. PSLF achieves effective multi-view feature fusion by distilling and exploiting consistent and complementary feature information from multiple views, while also selecting style patches from the candidates. Our style analysis approach supports both unsupervised and semi-supervised analysis. For the latter, our method accepts both user-specified shape labels and style-ranked triplets as clustering constraints. We demonstrate results from 3D shape style analysis andpatchlocalizationaswellasimprovementsoverstate-of-the-artmethods. We also present several applications enabled by our style analysis.


Overview


Our style co-analysis algorithm contains three stages: (a) Patch sampling and pre-selection and candidate style patches. (b) View feature encoding based on patch convolution, and (c) Multi-view feature integration using partially shared latent factor (PSLF) learning. The PSLF performs unsupervised or semi-supervised style clustering and patch filtering in an interleaving fashion.



Evaluation


Comparison on style classification between Hu et al. [2017] (blue) and our method (green) over datasets from their work




Visual comparison of style patches located by our method (right one in each pair) vs. those found by [Hu et al . 2017] (left one in each pair)





Applications


Style-aware mesh simplification. (a) Original meshes with style patches.(b-c)Shaded and wireframe versions of simplified models with style preservation via constrained quadric-based decimation; red boxes highlight significant triangle reduction near non-style areas. (d-e) Simplified models without style preservation, via unconstrained decimation.



Some examples of recognition and spatial localization of architec- tural styles on 3D building models. From the top to the bottom row: Asian, Byzantine, Gothic and Greece. For each example, the matched 2D style sketch is shown to the right. Style patches located on the 3D shapes are shown in orange color.





The source code and dataset can be found here
We have built a website for online testing our style recognition and style patchlocalization. You are welcomed to try out our method by uploading and testing your own 3D models
PAPER: Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines




Citation

BibTeX format:
@article{yu_tog18,
  title = {Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines},
  author = {Fenggen Yu and Yan Zhang and Kai Xu and Ali Mahdavi-Amiri and Hao Zhang},
  journal = {ACM Transactions on Graphics},
  volume = {37},
  number = {2},
  pages = {to appear},
  year = {2018}
}