Robust Multiobject Tracking Using Mmwave Radar-Camera Sensor Fusion

Abstract

With the recent hike in the autonomous and automotive industries, sensor-fusion-based perception has garnered significant attention for multiobject classification and tracking applications. Furthering our previous work on sensorfusion-based multiobject classification, this letter presents a robust tracking framework using a high-level monocularcamera and millimeter wave radar sensor-fusion. The proposed method aims to improve the localization accuracy by leveraging the radar’s depth and the camera’s cross-range resolutions using decision-level sensor fusion and make the system robust by continuously tracking objects despite single sensor failures using a tri-Kalman filter setup. The camera’s intrinsic calibration parameters and the height of the sensor placement are used to estimate a birds-eye view of the scene, which in turn aids in estimating 2-D position of the targets from the camera. The radar and camera measurements in a given frame is associated using the Hungarian algorithm. Finally, a tri-Kalman filter-based framework is used as the tracking approach. The proposed approach offers promising MOTA and MOTP metrics including significantly low missed detection rates that could aid large-scale and small-scale autonomous or robotics applications with safe perception.