| PhD Seminar

Name of the Speaker: Mr. Matta Gopi Raju (EE17D021)
Guide: Dr. Kaushik Mitra
Venue: ESB-244 (Seminar Hall)
Date/Time: 7th August 2024 (Wednesday), 3:00 PM
Title: Generalizable Neural Radiance Fields for Flare Removal

Abstract :

Flare, an optical phenomenon caused by unwanted scattering and reflections of light rays within a lens system, poses a significant challenge in imaging. Flares appear in diverse patterns (halos, streaks, color bleeding, haze, etc.), which makes flare removal challenging. Existing traditional and learning-based methods have exhibited limited efficacy due to their reliance on single-image approaches, where flare removal is highly ill-posed. Viewing flare removal as a multi-view image problem can enable the power to harness information from neighboring views, leveraging the fact that flare is view(direction) dependent. This underscores the notion that information that is lost by flare in one view can be extracted from alternate views. Building upon this concept, we propose GN-FR: Generalizable Neural Radiance Fields for Flare Removal, which can render flare-free novel views from a sparse set of input views that are corrupted by lens flare, and can also generalizes across scenes in an unsupervised setup. GN-FR includes the following modules: FMG (Flare-occupancy Mask Generation), VS (View sampler) , PS (Point sampler) embedded into the GNT framework. As capturing data both with and without flare is highly impractical, we propose a Masking loss function that incorporates mask information in an unsupervised fashion. We also contribute the first-of-its-kind 3D(multi-view) flare dataset, which contains 17 real flare scenes and also Flare pattern dataset. To the best of our knowledge, this is the first work that does flare removal in a NeRF setup. At the end, we also present our Novel View Synthesis(NVS) framework towards flare removal using 3D Gaussian Splatting(3DGS) technique.