다중 시점 RGB 영상의 심층 합성곱 신경망 기반 3차원 형상 추정 방법 = 3D Shape Estimation of Multiview RGB Images from Deep Convolutional Network

다국어 초록 (Multilingual Abstract)

In this paper, we describe point cloud generation method from multiple RGB images based on convolutional neural network. The proposed method is motivated from human’s 3d understanding from orthographic sketches, which infer spatial relationship from front, top, side view of the 3d object. The network of the proposed method employed generative model, to make it predict point clouds of atypical objects. Auto encoder network is utilized to encode three RGB images into latent vectors, and generate point clouds. Loss functions are defined which measure reconstruction performance and uniformity of the point clouds to make the network generate point clouds similar to the original and distribute uniformly along entire region. From this structure, we expected the network to learn spatial relation of the original 3d model from multiple RGB images. The result shows that the proposed method can predict overall shape of the objects, and it is hard to express detailed geometry. As a further work, network structure improvement to generate detailed shape of the object, predict occluded region will be performed.

다국어 초록 (Multilingual Abstract)

.

참고문헌 (Reference)

• 1 조재민 ; 강상승 ; 김계경, "비주얼 서보잉을 위한 딥러닝 기반 물체 인식 및 자세 추정" 한국로봇학회 14 (14): 1-7, 2019
• 2 김성흠 ; 복윤수 ; 권인소, "다시점 객체 공분할을 이용한 2D-3D 물체 자세 추정" 한국로봇학회 12 (12): 33-41, 2017
• 3 이재영 ; 조성진 ; 서용신 ; 박찬훈 ; 박동일 ; 김병인 ; 김휘수 ; 송성혁, "다공성 구조를 이용한 형상적응형 가변강성 소프트 그리퍼" 제어·로봇·시스템학회 27 (27): 238-246, 2021
• 4 SoftRobotics, inc, "mGrip"
• 5 R. Garg, "Unsupervised cnn for single view depth estimation: Geometry to the rescue" 740-756, 2016
• 6 김종형 ; 장경재 ; 권혁동, "Trinocular Vision System을 이용한 물체 자세정보 인식 향상방안" 한국생산제조학회 26 (26): 223-229, 2017
• 7 M. Ji, "Surfacenet: An endto-end 3d neural network for multiview stereopsis" 2307-2315, 2017
• 8 노유찬 ; 김영우 ; 김대건 ; 한현규 ; 송영기 ; 김덕수, "RGB-D 이미지 기반 물체 자세 인식 딥 러닝 파이프라인" 제어·로봇·시스템학회 27 (27): 593-601, 2021
• 9 L. Yu, "PU-net:Point cloud upsampling network" 2790-2799, 2018
• 10 R. Li, "PU-GAN:a point cloud upsampling adversarial network" 7203-7212, 2019
• 11 A. Kar, "Category-specific object reconstruction from a single image" 1966-1974, 2015
• 12 H. Fan, "A point set generation network for 3D object reconstruction from a single image" 2017
• 13 S. Tyree, "6-DoF pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark" 2019
• 14 C. r B. Choy, "3dr2n2:A unified approach for single and multi-view 3d object reconstruction" 628-644, 2016
• 15 Z. Lun, "3d shape reconstruction from sketches via multi-view convolutional networks" 67-77, 2017