Superpixel Segmentation Systems: Design and Analysis

Abstract

Image segmentation is a partitioning of an image into distinct groups of pixels (“regions”), each region preferably spatially connected, where each region ideally corresponds to an object or other semantically meaningful scene component. In this dissertation we discuss supervised evaluation of segmentation algorithms and the common types of error. One major part of the supervised evaluation framework is the need for ground-truth segmentations. With the growing popularity of supervised deep-learning techniques for semantic segmentation, there are large sets of labeled data. When these ground-truth datasets have multiple ground-truth segmentations for each image, it is important to choose the most appropriate ground-truth segmentation or a fusion of these ground-truth segmentations for an unbiased evaluation of segmentation algorithms. We introduce a new hybrid ground-truth fusion method, and we provide qualitative and quantitative results to show that our new method performs better than existing methods. Superpixel segmentation is an image segmentation in which each region (“superpixel”) preferably forms a portion of an object or scene component rather than the whole, where each superpixel is preferably homogeneous with respect to certain features (e.g., color or texture). Superpixel segmentation algorithms facilitate subsequent image analysis by reducing the number of image primitives from the order of 105 – 107 pixels to a few hundreds of superpixels. Thereby, the computation time is greatly reduced for further post-processing steps such as region merging for image segmentation, object tracking, detection, depth estimation, object recognition, image denoising, object-based compression, video processing, video coding, saliency detection, and deep learning. We present a four-way experimental evaluation of the effectiveness of measures used to quantify the performance of superpixel algorithms. Choosing the right superpixel evaluation measure will significantly improve the ability to evaluate superpixel segmentation algorithms, especially when the superpixel segmentation is being used as a pre-processing step for subsequent image analysis tasks. We determine the best superpixel evaluation measure through carefully designed experiments using five different datasets and six superpixel algorithms at four coarseness levels. In addition, we propose a new superpixel method called superpixels using morphology (SUM) for rock image segmentation. Qualitative results show that our method is comparable to other superpixel algorithms. From the quantitative results we observe that SUM has accuracy comparable to that of other algorithms. SUM, however, is the fastest among all the algorithms tested and simpler to implement. Further, we propose a new PCA-based image denoising method using superpixels. Our proposed method achieves very competitive denoising performance compared with many existing denoising algorithms with respect to both objective measurement and visual evaluation. We next discuss evaluation measures for use in a two-step segmentation process. We propose a new evaluation measure, F-entropy for use in a two-step segmentation process and conclude that F-entropy performs better as an evaluation measure based on experimental results. Finally, we present ideas for potential future work in this area.