ESR8 Fatemeh Ghorbani Lohesara

Fatemeh has been concentrating on the head-mounted display (HMD) removal problem in extended-reality (XR) applications. As a potential outcome of her research project, virtual communication can be improved by recovering the missing image information caused by the occlusion of an HMD in a realistic manner. Fatemeh collected a multimodal dataset (including multiview RGB-D, textured mesh, point cloud, and light field) through user experiments during her secondment at Tampere University (CIVIT Lab) to understand the usage of high-dimensional and real-world data in solving her research problem. She is also working on video inpainting algorithms designed based on generative adversarial networks (GANs) and applied to the collected data in XR telecommunication scenarios. Furthermore, a number of evaluation criteria are being considered in order to achieve a more accurate inpainted result as well as a photorealistic representation of facial expressions and eye gaze direction under HMD occlusion.