| PhD Seminar

Name of the Speaker: Joshitha R (EE19D701)
Guide: Dr. Mansi Sharma
Online meeting link: https://meet.google.com/cjj-ptqy-ewo
Date/Time: August 3 · 4:00 – 5:00pm
Title: Efficient Representation, Coding and Streaming of Light fields for Glasses-free 3D Displays

Abstract

Recently, significant progress has been made in developing autostereoscopic platforms for displaying real-world 3D scenes. Light fields are the best emerging choice for computational multi-view autostereoscopic displays since they provide an optimized solution to support direction-dependent outputs simultaneously without sacrificing the resolution. Such multi-view light field displays can accommodate numerous viewing directions, continuous motion parallax, greater depth-of-field and wider field-of-view. However, the primary challenge in handling light fields for any application are their enormous volume and requirements for storage and transmission. It is therefore critical to develop efficient representation and coding solutions for light fields suitable for display and streaming applications.

In this talk, we present two novel light field representations, coding and streaming schemes. The first is a hierarchical algorithm based on transmittance patterns of low-rank multiplicative layers and Fourier disparity layers for different light field scanning patterns. In another scheme, intrinsic redundancies in light field subsets of the same scanning patterns are eliminated through low-rank representation using Tucker decomposition with tensor sketching for various ranks and sketch dimension parameters. Both the proposed integrated formulations follow hierarchical scanning orders to operate on view subsets and analyze the approximated light field by sampling it in the depth dimension by decomposing the scene as a discrete sum of Fourier Disparity layers. This exploits additional intra-view, inter-view, and other redundancies among adjacent views in horizontal and vertical directions to further allow scalable light field coding.

Besides, the representation is appropriate for more general rendering by shifting approximated light field sub-aperture images in the depth dimension instead of two angular dimensions. We can reconstruct intermediate viewpoints from low-rank approximated views at different bitrates without any additional disparity maps. The proposed schemes are flexible to realize a range of multiple bitrates at the decoder within a single integrated system. Moreover, our hybrid low-rank approximation and encoding schemes can also be incorporated as a complement to other existing or future light field coding algorithms. Compression performance of the schemes analyzed on real light fields shows substantial bitrate savings compared to state-of-the-art codecs, while maintaining good reconstruction quality.