Siamese Cross-Domain Tracker Design for Seamless Tracking of Targets in RGB and Thermal Videos

Channappayya, Sumohana S. (2023) Siamese Cross-Domain Tracker Design for Seamless Tracking of Targets in RGB and Thermal Videos. IEEE Transactions on Artificial Intelligence, 4 (1). pp. 161-172. ISSN 2691-4581

[img] Text
ieee_tra_art_int_4-1.pdf - Published Version

Download (8MB)


Multimodal (RGB and thermal) applications are swiftly gaining importance in the computer vision community with advancements in self-driving cars, robotics, Internet of Things, and surveillance applications. Both the modalities have complementary performance depending on illumination constraints. Hence, a judicious combination of both modalities will result in robust RGBT systems capable of all-day all-weather applications. Several studies have been proposed in the literature for integrating the multimodal sensor data for object tracking applications. Most of the proposed networks try to delineate the information into modality-specific and modality shared features and attempt to exploit the modality shared features in enhancing the modality specific information. In this work, we propose a novel perspective to this problem using a Siamese inspired network architecture. We design a custom Siamese cross-domain tracker architecture and fuse it with a mean shift tracker to drastically reduce the computational complexity. We also propose a constant false alarm rate inspired coasting architecture to cater for real-time track loss scenarios. The proposed method presents a complete and robust solution for object tracking across domains with seamless track handover for all-day all-weather operation. The algorithm is successfully implemented on a Jetson-Nano, the smallest graphics processing unit (GPU) board offered by NVIDIA Corporation.

[error in script]
IITH Creators:
IITH CreatorsORCiD
Channappayya, Sumohana S.
Item Type: Article
Uncontrolled Keywords: Convolutional neural network (CNN); domain translation; generative adversarial network (GAN); mean-shift algorithm; Siamese networks; target tracking; Clutter (information theory); Computer graphics; Computer graphics equipment; Convolution; Graphics processing unit; Internet of things; Network architecture; Neural networks; Program processors; Security systems; Tracking radar; Convolutional neural network; Domain translation; GAN; Mean shift algorithm; Siamese network; Surveillance; Targets tracking; Video-tracking; Target tracking
Subjects: Electrical Engineering
Electrical Engineering > Electrical and Electronic
Divisions: Department of Electrical Engineering
Depositing User: Mr Nigam Prasad Bisoyi
Date Deposited: 22 Aug 2023 05:38
Last Modified: 22 Aug 2023 05:38
Publisher URL:
OA policy:
Related URLs:

Actions (login required)

View Item View Item
Statistics for RAIITH ePrint 11597 Statistics for this ePrint Item