Performance evaluation of automatic object detection with post-processing schemes under enhanced measures in wide-area aerial imagery

Abstract

Performance analysis of object detection combined with post-processing schemes are challenging especially that the spatial resolution of images is low in wide-area aerial imagery. In this paper, we present the quantitative results of ten object detection algorithms combined with several post-processing schemes including filtered dilation, heuristic filtering, sieving and closing, a three-stage scheme which involves thresholding with respect to area and compactness, and the proposed scheme of median filtering, opening and closing, followed by linear Gaussian filtering with nonmaximum suppression. We verified the sieving and closing as well as the three-stage scheme display better Fβ-score and PASCAL value via four vehicle detection algorithms. We evaluated combinations of ten object detection and segmentation methods with two post-processing schemes by adopting a set of recent evaluation metrics, i.e., Jaccard Index (JI), Fbw measure, the structure similarity measure (SSIM) and the enhanced alignment measure (EAM). Automatic detection outputs are compared with their ground truth in low-resolution aerial datasets. Classified detection results are established on ten algorithms each combined with the selected post-processing schemes. We take two widely used datasets (VIVID and VEDAI) for performance analysis, compare the detections and time cost of each algorithm either without or with the proposed scheme, and verified our approach via replacing either datasets or algorithms. Quantitative evaluation under a set of enhanced measures proves our test with validity, efficiency, and accuracy.