Cluttered Scene Segmentation Using the Symmetry Constraint

Abstract

Although modern object segmentation algorithms can deal with isolated objects in simple scenes, segmenting non-convex objects in cluttered environments remains a challenging task. We introduce a novel approach for segmenting unknown objects in partial 3D pointclouds that utilizes the powerful concept of symmetry. First, 3D bilateral symmetries in the scene are detected efficiently by extracting and matching surface normal edge curves in the pointcloud. Symmetry hypotheses are then used to initialize a segmentation process that finds points of the scene that are consistent with each of the detected symmetries. We evaluate our approach on a dataset of 3D pointcloud scans of tabletop scenes. We demonstrate that the use of the symmetry constraint enables our approach to correctly segment objects in challenging configurations and to outperform current state-of-the-art approaches.

Video

Published materials

A. Ecins, C. Fermüller, Y. Aloimonos.
Cluttered Scene Segmentation Using the Symmetry Constraint
International Conference on Robotics and Automation (ICRA), May 2016
[PDF] [Poster] [Slides (keynote)]

Sample results

Cluttered Tabletop Dataset

Clutterd Tabletop Dataset (CTD) contains 3D reconstructions of 89 scenes of various objects placed on a table. The set of objects includes simple objects like boxes as well as nonconvex objects such as a teddy bear. Complexity of the scenes varies from single objects to multiple objects put side by side and stacked on top of each other. Each pointcloud of the scene is annotated with per-object segmentation masks as well as per-object rotational and reflectional symmetries.
[Download] [Readme]