Hao Li (Pinscreen, UC Berkeley): “AI Synthesis for the Metaverse: From avatars to 3D scenes “

Speaker: Hao Li (Pinscreen, UC Berkeley)

Title: AI Synthesis for the Metaverse: From avatars to 3D scenes 

Abstract: As the world is getting ready for the metaverse, the need for 3D content is growing rapidly, AR/VR will become mainstream, and next era of the web will be spatial. A digital and immersive future is unthinkable without telepresence, lifelike digital humans, and photorealistic virtual worlds. Existing computer graphics pipelines and technologies rely on production studios and a content creation process that is time consuming and expensive. My research is about developing novel 3D deep learning-based techniques for generating photorealistic digital humans, objects, and scenes and democratizing the process by making such capability accessible to anyone and automatic. In this talk, I will present a state-of-the-art technology for digitizing an entire virtual 3D avatar from a single photo developed at Pinscreen, and give a live demo. I will also showcase a high-end neural rendering technology used in next generation virtual assistant solutions and real-time virtual production pipelines. I will also present a real-time teleportation system that only uses a single webcam as input for digitizing entire bodies using 3D deep learning. Furthermore, I will present our latest efforts at UC Berkeley on real-time AI synthesis of entire scenes using NeRF representations and Plenoctrees. Finally, I will highlight some recent work on deepfake detection and speech-driven human motion synthesis, where we combine approaches from NLP, vision and graphics. My goal is to enable new capabilities and applications at the intersection of AI, vision, and graphics and impact the future of communication, human-machine interaction, and content creation. At the same time, we must also prioritize the safety and wellbeing of everyone while architecting this future.

Bio: Hao Li is CEO and Co-Founder of Pinscreen, a startup that builds cutting edge AI-driven virtual avatar technologies. He is also a Distinguished Fellow of the Computer Vision Group at UC Berkeley. Before that, he was an Associate Professor of Computer Science at the University of Southern California, as well as the director of the Vision and Graphics Lab at the USC Institute for Creative Technologies. Hao’s work in Computer Graphics and Computer Vision focuses on digitizing humans and capturing their performances for immersive communication, telepresence in virtual worlds, and entertainment. His research involves the development of novel deep learning, data-driven, and geometry processing algorithms. He is known for his seminal work in avatar creation, facial animation, hair digitization, dynamic shape processing, as well as his recent efforts in preventing the spread of malicious deep fakes. He was previously a visiting professor at Weta Digital, a research lead at Industrial Light & Magic / Lucasfilm, and a postdoctoral fellow at Columbia and Princeton Universities. He was named top 35 innovator under 35 by MIT Technology Review in 2013 and was also awarded the Google Faculty Award, the Okawa Foundation Research Grant, as well as the Andrew and Erna Viterbi Early Career Chair. He won the Office of Naval Research (ONR) Young Investigator Award in 2018 and was named named to the DARPA ISAT Study Group in 2019. In 2020, he won the ACM SIGGRAPH Real-Time Live! “Best in Show” award. Hao obtained his PhD at ETH Zurich and his MSc at the University of Karlsruhe (TH).