Professor Christian Theobalt

Max-Planck-Institute (MPI) for Informatics

March 21, 2018 4:30 pm to 5:30 pm

Location: Packard 101

Talk Title: Video-based Reconstruction of the Real World in Motion

Talk Abstract: New methods for capturing highly detailed models of moving real world scenes with cameras, i.e., models of detailed deforming geometry, appearance or even material properties, become more and more important in many application areas. They are needed in visual content creation, for instance in visual effects, where they are needed to build highly realistic models of virtual human actors. Further on, efficient, reliable and highly accurate dynamic scene reconstruction is nowadays an important prerequisite for many other application domains, such as: human-computer and human-robot interaction, autonomous robotics and autonomous driving, virtual and augmented reality, 3D and free-viewpoint TV, immersive telepresence, and even video editing.

The development of dynamic scene reconstruction methods has been a long standing challenge in computer graphics and computer vision. Recently, the field has seen important progress. New methods were developed that capture - without markers or scene instrumentation - rather detailed models of individual moving humans or general deforming surfaces from video recordings, and capture even simple models of appearance and lighting. However, despite this recent progress, the field is still at an early stage, and current technology is still starkly constrained in many ways. Many of today's state-of-the-art methods are still niche solutions that are designed to work under very constrained conditions, for instance: only in controlled studios, with many cameras, for very specific object types, for very simple types of motion and deformation, or at processing speeds far from real-time.

In this talk, I will present some of our recent works on detailed marker-less dynamic scene reconstruction and performance capture in which we advanced the state of the art in several ways. For instance, I will briefly show new methods for marker-less capture of the full body (like our VNECT approach) and hands that work in more general environments, and even in real-time and with one camera. I will then show some of our work on high-quality face performance capture and face reenactment. Here, I will also illustrate the benefits of both model-based and learning-based approaches and show how different ways to join the forces of the two open up new possibilities. Live demos included !

More Information:

Speaker's Biography: Christian Theobalt is a Professor of Computer Science and the head of the research group "Graphics, Vision, & Video" at the Max-Planck-Institute (MPI) for Informatics, Saarbrücken, Germany. He is also a Professor of Computer Science at Saarland University, Germany.
From 2007 until 2009 he was a Visiting Assistant Professor in the Department of Computer Science at Stanford University.
He received his MSc degree in Artificial Intelligence from the University of Edinburgh, his Diplom (MS) degree in Computer Science from Saarland University, and his PhD (Dr.-Ing.) from Saarland University and Max-Planck-Institute for Informatics.

In his research he looks at algorithmic problems that lie at the intersection of Computer Graphics, Computer Vision and machine learning, such as: static and dynamic 3D scene reconstruction, marker-less motion and performance capture, virtual and augmented reality, computer animation, appearance and reflectance modelling, intrinsic video and inverse rendering, machine learning for graphics and vision, new sensors for 3D acquisition, advanced video processing, as well as image- and physically-based rendering. He is also interested in using reconstruction techniques for human computer interaction.

For his work, he received several awards, including the Otto Hahn Medal of the Max-Planck Society in 2007, the EUROGRAPHICS Young Researcher Award in 2009, the German Pattern Recognition Award 2012, and the Karl Heinz Beckurts Award in 2017. He received two ERC grants, one of the most prestigious and competitive individual research grants in Europe: An ERC Starting Grant in 2013 and an ERC Consolidator Grant in 2017. In 2015, he was elected as one of the top 40 innovation leaders under 40 in Germany by the business magazine Capital. Christian Theobalt is also a co-founder of an award-winning spin-off company from his group - - that is commercializing one of the most advanced solutions for marker-less motion and performance capture.


Professor Marty Banks

UC Berkeley

February 28, 2018 4:30 pm to 5:30 pm

Location: Packard 101

Talk Title: ChromaBlur: Rendering Chromatic Eye Aberration Improves Accommodation and Realism

Talk Abstract: Computer-graphics engineers and vision scientists want to generate images that reproduce realistic depth-dependent blur. Current rendering algorithms take into account scene geometry, aperture size, and focal distance, and they produce photorealistic imagery as with a high-quality camera. But to create immersive experiences, rendering algorithms should aim instead for perceptual realism. In so doing, they should take into account the significant optical aberrations of the human eye. We developed a method that, by incorporating some of those aberrations, yields displayed images that produce retinal images much closer to the ones that occur in natural viewing. In particular, we create displayed images taking the eye’s chromatic aberration into account. This produces different chromatic effects in the retinal image for objects farther or nearer than current focus. We call the method ChromaBlur. We conducted two experiments that illustrate the benefits of ChromaBlur. One showed that accommodation (eye focusing) is driven quite effectively when ChromaBlur is used and that accommodation is not driven at all when conventional methods are used. The second showed that perceived depth and realism are greater with imagery created by ChromaBlur than in imagery created conventionally. ChromaBlur can be coupled with focus-adjustable lenses and gaze tracking to reproduce the natural relationship between accommodation and blur in HMDs and other immersive devices. It can thereby minimize the adverse effects of vergence-accommodation conflicts.

Speaker's Biography: Martin S. Banks received his Bachelor’s degree from Occidental College in 1970. He majored in Psychology and minored in Physics. After spending a year in Germany teaching in their school system, he entered the graduate program at UC San Diego where he received a Master’s degree in Experimental Psychology in 1973. Banks then moved to the graduate program at the University of Minnesota where he received his Ph.D. in Developmental Psychology in 1976. He was Assistant and Associate Professor of Psychology at the University of Texas at Austin from 1976-1985. He moved to UC Berkeley School of Optometry in 1985 where he has been Associate and Full Professor of Optometry and Vision Science until the present. He was Chairman of the Vision Science Program from 1995-2002, and again in 2012.

Banks has received awards for his work on basic and applied research on human visual development, on visual space perception, and on the development and evaluation of visual displays. These include the Young Investigator Award from the National Research Council (1978), Boyd R. McCandless Award from the American Psychological Association (1984), Kurt Koffka Medal from Giessen University (2007), Charles F. Prentice Award from the American Academy of Optometry (2016), and Otto Schade Prize from the Society for Information Display (2017). He has also been appointed Fellow of the Center for Advanced Study of the Behavioral Sciences (1988), Fellow of the American Association for the Advancement of Science (2008), Fellow of the American Psychological Society (2009), Holgate Fellow of Durham University (2011), WICN Fellow of University of Wales (2011), Honorary Professor of University of Wales (2017), and Borish Scholar of Indiana University (2017).


Chris Metzler

Rice University

February 21, 2018 4:30 pm to 5:30 pm

Location: Packard 101

Talk Title: Data-driven Computational Imaging

Talk Abstract: Between ever increasing pixel counts, ever cheaper sensors, and the ever expanding world-wide-web, natural image data has become plentiful. These vast quantities of data, be they high frame rate videos or huge curated datasets like Imagenet, stand to substantially improve the performance and capabilities of computational imaging systems. However, using this data efficiently presents its own unique set of challenges. In this talk I will use data to develop better priors, improve reconstructions, and enable new capabilities for computational imaging systems.

Speaker's Biography: Chris Metzler is a PhD candidate in the Machine Learning, Digital Signal Processing, and Computational Imaging labs at Rice University. His research focuses on developing and applying new algorithms, including neural networks, to problems in computational imaging. Much of his work concerns imaging through scattering media, like fog and water, and last summer he interned in the U.S. Naval Research Laboratory's Applied Optics branch. He is an NSF graduate research fellow and was formerly an NDSEG graduate research fellow.


Professor Jacob Chakareski

University of Alabama

March 14, 2018 4:30 pm to 5:30 pm

Location: Packard 101

Talk Title: Drone IoT Networks for Virtual Human Teleportation

Talk Abstract: Cyber-physical/human systems (CPS/CHS) are set to play an increasingly visible role in our lives, advancing research and technology across diverse disciplines. I am exploring novel synergies between three emerging CPS/CHS technologies of prospectively broad societal impact, virtual/augmented reality (VR/AR), the Internet of Things (IoT), and autonomous micro-aerial robots (UAVs). My long-term research objective is UAV-IoT-deployed ubiquitous VR/AR immersive communication that can enable virtual human teleportation to any corner of the world. Thereby, we can achieve a broad range of technological and societal advances that will enhance energy conservation, quality of life, and the global economy.
I am investigating fundamental problems at the intersection of signal acquisition and representation, communications and networking, (embedded) sensors and systems, and rigorous machine learning for stochastic control that arise in this context. I envision a future where UAV-IoT-deployed immersive communication systems will break existing barriers in remote sensing, monitoring, localization and mapping, navigation, and scene understanding. The presentation will outline some of my present and envisioned investigations. Interdisciplinary applications will be highlighted.

Speaker's Biography: Jacob Chakareski is an Assistant Professor of Electrical and Computer Engineering at The University of Alabama, where he leads the Laboratory for VR/AR Immersive Communication (LION). His interests span virtual and augmented reality, UAV-IoT sensing and communication, and rigorous machine learning for stochastic control. Dr. Chakareski received the Swiss NSF Ambizione Career Award and the best paper award at ICC 2017. He trained as a PhD student at Rice and Stanford, held research appointments with Microsoft, HP Labs, and EPFL, and sits on the advisory board of Frame, Inc. His research is supported by the NSF, AFOSR, Adobe, NVIDIA, and Microsoft. For further info, please visit


Dr. Patrick Llull


March 7, 2018 4:30 pm to 5:30 pm

Location: Packard 101

Talk Title: Temporal coding of volumetric imagery

Talk Abstract: 'Image volumes' refer to realizations of images in other dimensions such as time, spectrum, and focus. Recent advances in scientific, medical, and consumer applications demand improvements in image volume capture. Though image volume acquisition continues to advance, it maintains the same sampling mechanisms that have been used for decades; every voxel must be scanned or captured in parallel and is presumed independent of its neighbors. Under these conditions, improving performance comes at the cost of increased system complexity, data rates, and power consumption.

This talk describes systems and methods with which to efficiently detect and visualize image volumes by temporally encoding the extra dimensions’ information into 2D measurements or displays. Some highlights of my research include video and 3D recovery from photographs, and true-3D augmented reality image display by time multiplexing. In the talk, I show how temporal optical coding can improve system performance, battery life, and hardware simplicity for a variety of platforms and applications.

Speaker's Biography: Currently with Google's Daydream virtual reality team, Patrick Llull completed his Ph.D. under Prof. David Brady at the Duke University Imaging and Spectroscopy Program (DISP) in May 2016. His doctoral research focused on compressive video and multidimensional sensing, with research internship experience with Ricoh Innovations in near-eye multifocal displays. During his Ph.D. Patrick won two best paper awards and was an NSF graduate fellowship honorable mention. Patrick graduated with his BS from the University of Arizona's College of Optical Sciences in May 2012.


Previous SCIEN Colloquia

To see a list of previous SCIEN colloquia, please click here.