| PhD Viva

Name of the Speaker: Ms. Nisha Varghese (EE19D750)
Guide: Dr. Rajagopalan AN
Online meeting link: https://meet.google.com/ijh-cafz-dsb
Date/Time: 24th June 2025 (Tuesday), 11 AM
Title: Enhanced visual understanding for terrestrial and underwater scenes

Abstract :

This work presents frameworks for enhanced visual understanding in terrestrial and underwater environments. In terrestrial scenes, by utilizing multi-camera systems with wide field-of-view (FOV) cameras, our approach enables continuous tracking of multiple moving point targets. We also address the motion deblurring in gimbal-based imaging systems, which can be used to expand the coverage of the visually monitored area. For underwater (UW) scenarios, the proposed method enhances visual clarity through effective image restoration and depth estimation in both well-lit and low-light conditions.

Multiple point target tracking is important in surveillance and security applications to issue early warnings of approaching threatening targets. To enhance the scene understanding, a multi-camera system can be used to track the point targets, which is more effective due to the extensive coverage. First, to track point targets in a single camera, we propose a detection and a systematic track association method. For tracking targets across multiple cameras, a proper data association for point targets is required, which is an unexplored research area. We propose the first-ever data association scheme for multiple-point target tracking across multiple cameras, utilizing epipolar geometry and the 3D angles of the point targets. We also address the lens distortion problem in the wide field-of-view (FOV) cameras used in such tracking applications. To accurately calibrate a wide-FOV camera, we obtained data from an existing experimental setup with a pan-tilt system and a fixed-point thermal source, as traditional methods using calibration patterns are unsuitable for wide-FOV cameras, and deep learning-based approaches lack generalizability to unseen scenes. Next, we focus on improving visual perception of terrestrial scenes using camera-gimbal systems that undergo fast to-and-fro rotational motion to surveil an extended field-of-view. Fast motion of the gimbal results in significant motion blur in the captured images. In this work, we address the problem of real-time motion deblurring in images captured by a real gimbal-based system. We propose two blur-kernel estimation methods and reveal how a priori knowledge of the blur-kernel can be used in conjunction with non-blind deblurring methods to achieve real-time performance. We also propose a Gyro-guided deep learning network for real-time motion deblurring, which has generalization capability.

UW images often suffer from low contrast and contain inaccurate colors due to wavelength-dependent light scattering and absorption in water. For the enhanced visual understanding of the UW environment, depth estimation and image restoration from UW images is an important precursor. By harnessing the physical model for UW image formation in conjunction with the view-synthesis constraint on neighboring frames, we propose a self-supervised network that simultaneously outputs the depth map and the restored image in real-time. Extensive experiments on different UW datasets establish the superiority of our proposed method over prior art. Most of the UW image restoration and depth estimation methods including ours have been devised for images under normal lighting. Consequently, they struggle to perform on poorly lit UW images. Hence, we extend our work on normally-lit UW images to low-light UW (LLUW) images by incorporating the Retinex model and the stereo image geometry into our previous framework. For training this network, we have collected a UW dataset with stereo pairs of low-light and normally-lit UW images. Here, we also propose a masking scheme to prevent dynamic transients in monocular frames from misguiding the learning process. Evaluations on different LLUW datasets demonstrate the superiority and generalization ability of our proposed method over existing state-of-the-art methods.