2020 SCIEN Affiliates Meeting Poster Presentations
Index to Posters
Neural Holography with Camera-in-the-loop Training by Yifan (Evan) Peng, Suyeon Choi, Nitish Padmanaban, Jonghyun Kim, Gordon Wetzstein
Gaze-contingent stereo rendering for improving depth perception in augmented and virtual reality by Brooke Krajancich, Petr Kellnhofer, Gordon Wetzstein
Sub-millisecond pupil tracking with micron precision by Bartlomiej Kowalski and Alfredo Dubra
3D Imaging Through Scattering Media based on Confocal Diffuse Tomography by David Lindell, Gordon Wetzstein
Michelson Holography: Dual-SLM Holography with Camera-in-the-loop Optimization by Suyeon Choi, Jonghyun Kim, Yifan Peng, Gordon Wetzstein
D-VDAMP: Denoising-based Approximate Message Passing for Compressive MRI by Christopher Metzler, Gordon Wetzstein
SUREMap: Predicting Uncertainty in CNN-based Image Reconstruction Using Stein's Unbiased Risk Estimate by Kao Kitichotkul, Christopher Metzler, Frank Ong, Gordon Wetzstein
Depth from Learned Defocus by Hayato Ikoma, Cindy Nguyen, Evan Peng, Christopher Metzler, Gordon Wetzstein
Modeling a camera designed to measure fluorescence in oral cancer screening by Zheng Lyu, Haomiao Jiang, Brian Wandell, Joyce Farrell
Neural Implicit Non-line-of-sight Imaging by Mark Nishimura, Joshua Rapp, David B. Lindell, Sean I. Young, Gordon Wetzstein
Nanosecond Imaging on CMOS Cameras for Fluorescence Lifetime Microscopy and LIDAR by Adam Bowman and Mark Kasevich
Shared Anatomy Experience in Mixed Reality using Microsoft HoloLens by Chloe Huang, Christoph Leuze
Solving PDEs using generalized implicit neural representations by Qingqing Zhao, David Lindell, Cindy My Anh Nguyen, Gordon Wetzstein
Implicit Neural Representations with Periodic Activation Functions by Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein
Perfect RGB-IR color routers for sub-wavelength size CMOS image sensor pixels by Nathan Zhao, Peter Catrysse, Shanhui Fan
ISETAuto: Detecting cars with depth and radiance information by Zhenyi Liu, Joyce Farrell, Brian Wandell
Generative Network Based Optimization of Photonic Devices with Hard Constraints by Jiaqi Jiang, Mingkun Chen, Jonathan A. Fan
Image degradation due to dynamic wavefront distortion in resonant scanners by Vyas Akondi, Bartlomiej Kowalski, Stephen A. Burns, Alfredo Dubra
Correction of Resonant Scanner Dynamic Aberration by Xiaojing Huang, Julie Bentley, Alfredo Dubra
Cinematic Virtual Reality with Head-Motion Parallax by Jayant Thatte and Bernd Girod
Abstracts
Title: Neural Holography with Camera-in-the-loop Training
Authors: Yifan (Evan) Peng, Suyeon Choi, Nitish Padmanaban, Jonghyun Kim, Gordon Wetzstein
Abstract: Holographic displays promise unprecedented capabilities for direct-view displays as well as virtual and augmented reality applications. However, one of the biggest challenges for computer-generated holography (CGH) is the fundamental tradeoff between algorithm runtime and achieved image quality, which has prevented high-quality holographic image synthesis at fast speeds. Moreover, the image quality achieved by most holographic displays is low, due to the mismatch between the optical wave propagation of the display and its simulated model. Here, we develop an algorithmic CGH framework that achieves unprecedented image fidelity and real-time framerates. Our framework comprises several parts, including a novel camera-in-the-loop optimization strategy that allows us to either optimize a hologram directly or train an interpretable model of the optical wave propagation and a neural network architecture that represents the first CGH algorithm capable of generating full-color high-quality holographic images at 1080p resolution in real time.
Bio: Yifan (Evan) Peng is a Postdoctoral Research Fellow at Electrical Engineering, Stanford University. His research interest rides across the interdisciplinary fields of optics/photonics, computer graphics, and computer vision. Much of his work concerns computational imaging modalities combining optics and algorithms, involving both computational camera and display ends. He completed his Ph.D. in Computer Science at the University of British Columbia, and his M.Sc. and B.E. in Optical Science and Engineering at Zhejiang University.
Title: Gaze-contingent stereo rendering for improving depth perception in augmented and virtual reality
Authors: Brooke Krajancich, Petr Kellnhofer, Gordon Wetzstein
Abstract: Virtual and augmented reality (VR/AR) displays crucially rely on stereoscopic rendering to enable perceptually realistic user experiences. Yet, existing near-eye display systems ignore the gaze-dependent shift of the no-parallax point in the human eye. Here, we introduce a gaze-contingent stereo rendering technique that models this effect and conduct several user studies to validate its effectiveness. Our findings include experimental validation of the location of the no-parallax point, which we then use to demonstrate significant improvements of disparity and shape distortion in a VR setting, and consistent alignment of physical and digitally rendered objects across depths in optical see-through AR. Our work shows that gaze-contingent stereo rendering improves perceptual realism and depth perception of emerging wearable computing systems.
Bio: Brooke is a third year electrical engineering PhD candidate working in the Computational Imaging Lab at Stanford, supervised by Professor Gordon Wetzstein. Her research focuses on developing new algorithms and systems for enhancing the perceptual realism of current-generation virtual and augmented reality displays.
Title: Sub-millisecond pupil tracking with micron precision
Authors: Bartlomiej Kowalski and Alfredo Dubra
Abstract: We describe a monocular pupil tracker designed for eye motion stabilization consisting of a pair of 940 nm light-emitting diodes, a near-infrared CMOS camera with Camera Link interface, a field-programmable gate array (FPGA) and a central processing unit (CPU) is presented. Background subtraction, field-flattening, thresholding, pupil edge detection and outlier discarding are all performed on an FPGA pixel stream, while the fitting of the pupil edge to an ellipse is performed on the CPU, achieving up to 2,000 frames/s (camera-limited) with sub-ms latency and ~5-10 um precision were achieved.
Bio: Bartlomiej (Bartek) Kowalski is a computer engineer with a Master's degree from the Czestochowa University of Technology in Poland. He has four years of experience working with Canon developing fully automated optical coherence tomographer for retinal imaging. Since joining the Dubralab at Stanford five years ago, he has developed a new generation of image acquisition software for multi-spectral and multiple-scattering imaging with point- and line-scanning instruments, as well as the FPGA-CPU pupil tracking described in this poster.
Title: 3D Imaging Through Scattering Media based on Confocal Diffuse Tomography
Authors: David Lindell, Gordon Wetzstein
Abstract: Optical imaging techniques, such as light detection and ranging (LiDAR), are essential tools in remote sensing, robotic vision, and autonomous driving. However, the presence of scattering places fundamental limits on our ability to image through fog, rain, dust, or the atmosphere. Conventional approaches for imaging through scattering media operate at microscopic scales or require a priori knowledge of the target location for 3D imaging. We introduce a technique that co-designs single-photon avalanche diodes, ultra-fast pulsed lasers, and a new inverse method to capture 3D shape through scattering media. We demonstrate acquisition of shape and position for objects hidden behind a thick diffuser (≈6 transport mean free paths) at macroscopic scales. Our technique, confocal diffuse tomography, may be of considerable value to the aforementioned applications.
Bio: David Lindell (https://davidlindell.com) is a PhD candidate in the Department of Electrical Engineering at Stanford University, advised by Gordon Wetzstein. His research interests are in the areas of computational imaging, remote sensing, and machine learning. Recently, he has worked on developing advanced 3D imaging systems to capture objects hidden around corners or through scattering media.
Title: Michelson Holography: Dual-SLM Holography with Camera-in-the-loop Optimization
Authors: Suyeon Choi, Jonghyun Kim, Yifan Peng, Gordon Wetzstein
Abstract: We introduce Michelson Holography (MH), a holographic display technology that optimizes image quality for emerging holographic near-eye displays. Using two spatial light modulators, MH is capable of leveraging destructive interference to optically cancel out undiffracted light corrupting the observed image. We calibrate this system using emerging camera-in-the-loop holography techniques and demonstrate state-of-the art holographic 2D image quality.
Bio: Suyeon Choi is a first-year Ph.D. student at Stanford University in Electrical Engineering, advised by Prof. Gordon Wetzstein. His research interests include computational displays, algorithmic frameworks for optical systems, and light transport. He completed his bachelor's degree in Electrical and Computer Engineering at Seoul National University in 2019.
Title: D-VDAMP: Denoising-based Approximate Message Passing for Compressive MRI
Authors: Christopher Metzler, Gordon Wetzstein
Abstract: Plug and play (P&P) algorithms iteratively apply highly optimized image denoisers to impose priors and solve computational image reconstruction problems, to great effect. However, in general the "effective noise", that is the difference between the true signal and the intermediate solution, within the iterations of P&P algorithms is neither Gaussian nor white. This fact makes existing denoising algorithms suboptimal. In this work, we propose a CNN architecture for removing colored Gaussian noise and combine it with the recently proposed VDAMP algorithm, whose effective noise follows a predictable colored Gaussian distribution. We apply the resulting denoising-based VDAMP (D-VDAMP) algorithm to variable density sampled compressive MRI where it substantially outperforms existing techniques.
Bio: Chris Metzler is an Intelligence Community Postdoctoral Research Fellow in the Stanford Computational Imaging Lab and will join the University Maryland Computer Science Department in January. Previously, he was an NSF Graduate Research Fellow, a DoD NDSEG Fellow, and a NASA Texas Space Grant Consortium Fellow in the Digital Signal Processing and Computational Imaging Labs at Rice University. His research develops data-driven solutions to challenging imaging problems.
Title: SUREMap: Predicting Uncertainty in CNN-based Image Reconstruction Using Stein's Unbiased Risk Estimate
Authors: Kao Kitichotkul, Christopher Metzler, Frank Ong, Gordon Wetzstein
Abstract: Convolutional neural networks (CNN) have emerged as a powerful tool for solving computational imaging reconstruction problems. However, CNNs are generally difficult-to-understand black-boxes. Accordingly, it is challenging to know when they will work and, more importantly, when they will fail. This limitation is a major barrier to their use in safety-critical applications like medical imaging: Is that blob in the reconstruction an artifact or a tumor? In this work we use Stein's unbiased risk estimate (SURE) to develop per-pixel confidence intervals, in the form of heatmaps, for compressive sensing reconstruction using the approximate message passing (AMP) framework with CNN-based denoisers. These heatmaps tell end-users how much to trust an image formed by a CNN, which could greatly improve the utility of CNNs in various computational imaging applications
Bio: Ruangrawee "Kao" Kitichotkul is an undergraduate student majoring in electrical engineering at Stanford University. His fields of interest include signal processing and computational imaging.
Title: Depth from Learned Defocus
Authors: Hayato Ikoma, Cindy Nguyen, Evan Peng, Christopher Metzler, Gordon Wetzstein
Abstract: Depth estimation is challenging but essential for many advanced computer vision tasks. Among various approaches, depth-from-defocus (DfD) methods use defocus cues to recover depth. Recently, deep neural networks have been applied to simultaneously use defocus cues and advanced its accuracy. Furthermore, end-to-end optimization of optics and image processing is applied to find a better defocus blur for DfD methods. However, the previous attempts use linear optical image formation models and do not consider a crucial depth cue: occlusion. In this work, we use an occlusion-aware nonlinear image formation model and employ a radially symmetric optical model to efficiently optimize the blur. We show that our method achieves superior performance to previous works in simulation as well as in a custom prototype.
Bio: Hayato Ikoma is a Ph.D. student at Department of Electrical Engineering, Stanford University and a member of Stanford Computational Imaging Group. For the Ph.D. research, he is focusing on the development of computational imaging techniques for fluorescence optical microscopy and broadly interested in signal processing, machine learning and optimization. Most of his recent works focus on the end-to-end optimization of optical imaging systems, which involves machine learning and micro fabrication of an optical element. He also serves as a teaching assistant for EE267: Virtual Reality. He is expected to graduate at the end of March and is actively looking for a full-time job.
Cindy Nguyen is a second-year Electrical Engineering PhD student in the Stanford Computational Imaging Lab, advised by Gordon Wetzstein. Her research interests include end-to-end optimization and 3D imaging.
Title: Modeling a camera designed to measure fluorescence in oral cancer screening
Authors: Zheng Lyu, Haomiao Jiang, Brian Wandell, Joyce Farrell
Abstract: We are using image systems simulation technology to design digital cameras for measuring fluorescent signals; a first application is oral cancer screening. We describe the software tools that enable us to model the end-to-end system, beginning with the three-dimensional scene spectral radiance, including the optics, and deriving sensor output. We validate the simulations by comparing real and simulated camera image sensor data for calibrated color targets and for tongues in healthy individuals. Validation of the simulations with respect to these measurements increases our confidence that the stimulation tools are useful for designing novel cameras to quantitatively measure the fluorescence and support screening.
Bio: Zheng Lyu is a PhD candidate in the Department of Electrical Engineering at Stanford University, advised by Prof. Brian Wandell and Dr. Joyce Farrell. His research interests focus on camera and imaging system simulation and design. Zheng’s recent work involves camera modeling for fluorescence modeling and camera model validation based on 3D scenes.
Title: Neural Implicit Non-line-of-sight Imaging
Authors: Mark Nishimura, Joshua Rapp, David B. Lindell, Sean I. Young, Gordon Wetzstein
Abstract: The ability to reconstruct shape information beyond a camera's direct line of sight has potential applications ranging from search-and-rescue to autonomous driving. In transient non-line-of-sight imaging, a hidden scene is reconstructed from time-resolved measurements of light that has undergone multiple reflections. While fast and efficient reconstruction techniques exist, they are limited in resolution by their discrete, explicit representations of 3D geometry and typically assume linear image formation models. Building off recent work in neural implicit representations, we model the hidden scene with a signed distance function defined using a neural network, and propose a transient rendering pipeline that enables an analysis-by-synthesis reconstruction. Our method is highly accurate, uses a continuous representation of the hidden volume, and even reconstructs partially-occluded geometry. We evaluate our method in simulation and on real-world data, and demonstrate substantial improvements over existing methods.
Bio: Mark Nishimura is a third-year PhD student in the Computational Imaging Lab. He is interested in 3D imaging with deep learning, sensor fusion, and single photon imaging techniques.
Joshua Rapp is currently a postdoctoral researcher in the Computational Imaging Lab. He received his Ph.D. degree in electrical engineering from Boston University in 2020. His research interests include computational imaging, lidar, and statistical signal processing.
Title: Nanosecond Imaging on CMOS Cameras for Fluorescence Lifetime Microscopy and LIDAR
Authors: Adam Bowman and Mark Kasevich
Abstract: We demonstrate techniques for wide-field nanosecond imaging on any camera. Electro-optic modulators are re-imagined as imaging components, enabling fast image gating while keeping the favorable performance of slow CMOS and CCD sensors. Polarizing beamsplitters convert time-varying polarization to intensity in spatially separated image frames, encoding parameters like the fluorescence lifetime or time-of-flight from a scene. Using this approach, we developed a wide-field fluorescence lifetime microscope capable of observing single-molecule dynamics at high frame rate with 40 MHz image gating. More than 5 orders of magnitude improvement in throughput is achieved compared to single photon counting. Electro-optic imaging avoids many of the limitations of other wide-field detectors - including low photon efficiency, high noise, pile-up, and saturation. We are currently developing compact, low-cost systems for fluorescence and flash LIDAR applications.
Bio: I am a 4th year PhD student in Applied Physics working in Mark Kasevich's lab on building a multi-pass quantum electron microscope. I initially developed electro-optic imaging as a side project in the lab, and have since become very excited about its potential applications. We have applied for two patents on this technology.
Title: Shared Anatomy Experience in Mixed Reality using Microsoft HoloLens
Authors: Chloe Huang, Christoph Leuze
Abstract: In light of the COVID-19 pandemic, working from home has become the new normal. However, medical work is particularly challenging to conduct remotely due to its collaborative, hands-on, and meticulous nature. This research project aims to create a solution that leverages current technologies to help surgeons and other medical professionals continue their work from anywhere.
With a goal of using mixed reality (MR) to emulate the in-person medical work and learning experience remotely, we drew inspiration from Case Western Reserve University and Cleveland Clinic’s HoloAnatomy, a shared holographic experience in MR that enhances the anatomy learning experience. For this project, we aim to create a synchronous and shared experience that can be accessed remotely, in which the human anatomy can be explored by various people present.
The experience was developed with Unity Engine, C#, and Microsoft’s Mixed Reality Toolkit for the Microsoft HoloLens headset. For remote connectivity, we created a server in Photon, a network engine and multiplayer platform, with their Unity Networking plugin that all users connect to upon joining the experience.
Once users join, an interactive human body model is placed in their proper space. All other users can be located in relation to the body, represented by a HoloLens model. Each user is equipped with a voice-activated laser pointer that follows their gaze for communication. The movement of the body in-scene is controlled by the master client, while all users can use voice command phrases to show or hide various anatomy parts.
Nowadays, remote solutions are necessary for nearly all aspects of life. We were therefore motivated to create a versatile project, focusing on remote connectivity technology. With the human body implementation, the need for physical cadavers are eliminated, and the scene could be used in remote experiences from medical conference presentations to surgical practices. Doctors could even implement patient-specific 3D models, and add any features they see fit. Ultimately, professionals will be given the opportunity to collaborate in the same space remotely, which is a powerful tool that transcends distance.
Bio: Chloe Huang is a multilingual undergraduate at Stanford University studying Product Design Engineering and Human Computer Interaction. Working at the intersection of creativity and technology, she is passionate about designing innovative solutions to improve people's lives.
Title: Solving PDEs using generalized implicit neural representations
Authors: Qingqing Zhao, David Lindell, Cindy My Anh Nguyen, Gordon Wetzstein
Abstract: Partial differential equations are of great interest in many research fields like physics, engineering, and finance. However, finding their solutions quickly and accurately remains a computational challenge. On the other hand, recent developments in implicit neural representations have demonstrated that MLPs with sinusoidal activation functions (SIREN) is capable of representing complex natural signals with high granularity. In this work, we propose to solve classes of PDEs by learning the space of implicit neural representations of the PDE solutions conditioning on any parametric dependencies. We integrate the conditioning information into SIREN using conditional affine transformation, i.e. scaling and biasing the hidden layers based on the conditioning information. We perform experiments on 2D Helmholtz equations and demonstrate that our approach outperforms conventional generalization methods like hypernetwork, yields fast solutions with high accuracy.
Bio: Qingqing Zhao is a Ph.D. student in Electrical Engineering advised by Prof. Wetzstein.
Title: Implicit Neural Representations with Periodic Activation Functions
Authors: Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein
Abstract: Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations. However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail. They also fail to accurately model spatial and temporal derivatives, which is necessary to represent signals defined implicitly by differential equations. We propose to leverage periodic activation functions for implicit neural representations and demonstrate that these networks, dubbed sinusoidal representation networks or SIRENs, are ideally suited for representing complex natural signals and their derivatives. We analyze SIREN activation statistics to propose a principled initialization scheme and demonstrate the representation of images, wavefields, video, sound, three-dimensional shapes, and their derivatives. Further, we show how SIRENs can be leveraged to solve challenging boundary value problems, such as particular Eikonal equations (yielding signed distance functions), the Poisson equation, and the Helmholtz and wave equations. Lastly, we combine SIRENs with hypernetworks to learn priors over the space of SIREN functions. Please see the project website for a video overview of the proposed method and all applications (https://vsitzmann.github.io/siren/).
Bio: Julien Martel is a Postdoctoral Research Fellow in the Stanford Computational Imaging lab. Alexander Bergman is a third year PhD student in the Stanford Computational Imaging lab.
Title: Perfect RGB-IR color routers for sub-wavelength size CMOS image sensor pixels
Authors: Nathan Zhao, Peter Catrysse, Shanhui Fan
Abstract: High resolution image sensing technologies have exploded over the past decade. A critical capability of all image sensors is to separate light into its individual color components. In most technologies today, this is done via color filters. Filters, however, intrinsically waste a large fraction of the light by absorption or scattering. This affects image sensor performance since the amount of light incident on each image sensor pixels reduces quadratically with linear scaling of pixel size. This is particularly detrimental to the performance of (sub-)wavelength size pixels. In this paper, we provide a conceptually novel approach for color functionality in image sensors, by designing a color router that achieves perfect RGB-IR color routing for sub-wavelength size pixels. In a color router, all incident light for each color channel is routed directly and without loss to the photodetector of the corresponding color channel pixel. We show that color routers can be designed to near-perfectly match a prescribed spectral shape, which is important for color image processing. We further show that we can design these routers to achieve specific spectral bandwidth and to meet angular as well as fabrication constraints.
Bio: Nathan Zhao is a fifth year PhD Student in the department of Applied Physics. His research area is photonics/numerical methods.
Title: ISETAuto: Detecting cars with depth and radiance information
Authors: Zhenyi Liu, Joyce Farrell, Brian Wandell
Abstract: Two classes of devices - depth sensing LiDAR and radiance sensing cameras - are being used in autonomous driving applications. We use image systems simulation and real-world data to estimate the performance of a ResNet for car detection in complex scenes when the input is a depth image (Z = d(x,y)), a radiance image (L = r(x,y)), or both. We report three analyses. (1) When the spatial sampling (x,y) of the depth and radiance images are equated at typical camera resolutions, a ResNet detects cars at higher average precision from depth than radiance data. (2) As the spatial sampling resolution of the depth image declines to the range of current LIDAR devices, the ResNet average precision is lower with depth than radiance data. (3) We simulate a hybrid system that merges depth and radiance information by replacing data in the blue color channel with a depth image at the current LiDAR spatial resolution (RGD). The ResNet average precision on the RGD data is better than using depth or radiance alone. We confirmed these simulations using real-world data provided by Waymo. For the low spatial resolution of current LiDAR devices, ResNet average precision of car detection is improved by using the combined (RGD) data as input.
Bio: Zhenyi Liu is a 5th year Ph.D. candidate from Jilin University and he was a visiting student in Prof. Brian Wandell’s lab. His research involves developing methods of generating physically accurate virtual scenes that are used in simulations to evaluate camera and lidar sensors for automotive applications.
Title: Generative Network Based Optimization of Photonic Devices with Hard Constraints
Authors: Jiaqi Jiang, Mingkun Chen, Jonathan A. Fan
Abstract: Inverse design algorithms are the basis for realizing high-performance photonic devices. Recently we have developed generative neural networks based global optimization algorithm (GLOnets) which are good at high-dimensional non-convex optimization. GLOnets are gradient-based optimizers that do not use a training set but instead combine a generative neural network with an electromagnetic simulator to perform population-based optimization. However, incorporating hard constraint in the GLOnets framework remains a challenge. In this work, we show that hard constraints can be imposed on inverse-designed devices by reparameterizing the design space itself. Instead of evaluating and modifying devices in the physical device space, candidate device layouts are defined in a constraint-free latent space and mathematically transformed to the physical device space. We apply this optimization concept to two photonic systems. One is metagratings with strict minimum feature size constraints for fabrication considerations, and the other is multi-layer optical thin-film stacks where refractive index of each layer is restricted to be a categorical variable. Benchmarks with known global optimization algorithms indicate that GLOnets can find the global optimum more accurately and efficiently.
Bio: Jiaqi Jiang is a 4th year PhD student in the department of Electrical Engineering, working in Prof. Jonathan Fan's lab. His research interests focus on inverse design of photonic systems and application of deep learning in physical science.
Mingkun Chen is a 3rd year PhD student in the department of Electrical Engineering, working in Prof. Jonathan Fan's lab.
Title: Image degradation due to dynamic wavefront distortion in resonant scanners
Authors: Vyas Akondi, Bartlomiej Kowalski, Stephen A. Burns, Alfredo Dubra
Abstract: Resonant galvanometric optical scanners are high-speed optical devices that are revolutionizing imaging in microscopy, DNA sequencing, flow cytometry, as well as numerous other applications, including data storage, display technologies, printing, and autonomous vehicles. We show that the dynamic wavefront distortion of these devices causes a substantial loss of transverse image resolution. The repeatability of the dynamic distortion measurements indicate that computational and optical corrective methods can be used to restore transverse resolution in imaging applications.
Bio: Vyas Akondi is a senior research scientist in the Dubra lab with interests in wavefront sensing and adaptive optics for retinal imaging. He obtained his Ph.D. degree from the Indian Institute of Science in Bangalore, India, and before joining the Dubra lab at Stanford, he pursued postdoctoral research in two different labs led by Prof. Susana Marcos and Prof. Brian Vohnsen.
Title: Correction of Resonant Scanner Dynamic Aberration
Authors: Xiaojing Huang, Julie Bentley, Alfredo Dubra
Abstract: Resonant optical scanners enable a diverserange of high speed and high resolution optical technologiesin biomedical imaging, microscopy, DNA sequencing, flow cytometry, data storage, display technologies, printing, and autonomous vehicles. We recently showed that the dynamic mirror distortion in these devices can result in a substantial degradation of transverse resolution, and in confocal systems, also a reduction in signal. Here, we demonstrate a practical and low-costsolution that only requires tilting of existing optical elements immediately followingthe resonant scanner.This solution is based on nodal aberration theory, a most elegant and insightful mathematical description of optical systems with tilted and decentered surfaces.The proposed method can be used to generate any desired third order aberration that results from tilting or decentering optical surfaces in optical systems with any pupil or field of view geometry. This is illustrated by correctinglinear astigmatismdue to a resonant scanner by tilting elements in reflective and refractive afocal relaysfor a single vergence, as well as a vergence range. In all cases, wavefront correction better than the classical diffraction limit(Strehl ratio > 0.8)was demonstrated,with and without the use of a adaptive wavefront corrector such as a deformable mirror.
Bio: Xiaojing (Grayce) Huang is currently an optics Ph.D. student working with Professors Julie Bentley at the University of Rochester and Alfredo Dubra at Stanford University. She holds a bachelor’s degree in Optical Engineering and a minor in Japanese from the University of Rochester. She is interested in design of imaging systems, nodal aberration theory and metrology. Her current research focuses on advancing adaptive optics ophthalmoscopy from a research to a clinical technology.
Title: Cinematic Virtual Reality with Head-Motion Parallax
Authors: Jayant Thatte and Bernd Girod
Abstract: Even as virtual reality has rapidly gained popularity over the past decade, visual fatigue, imperfect sense of immersion, and nausea remain significant barriers to its wide adoption. A key cause of this discomfort is the failure of the current technology to render accurate perspective changes or parallax resulting from the viewer’s head-motion. This mismatch induces a visual-vestibular conflict. This work addresses the issue by proposing an end-to-end framework that can capture, store, and render natural scenery with accurate head-motion parallax.
At the core of the problem is the trade-off between storing enough scene information to facilitate fast, high-fidelity rendering of head-motion parallax and keeping the representation compact enough to be practically viable. In this regard, we explore several novel scene representations, compare them with qualitative and quantitative evaluations, and discuss their advantages and disadvantages. We demonstrate the practical applicability of the proposed representations by developing an end-to-end virtual reality system that can render real-time head-motion parallax for natural environments. To that end, we build a two-level camera rig and present an algorithm to construct the proposed representations using the images captured by our camera system. Furthermore, we develop a custom OpenGL renderer that uses the constructed intermediate representations to synthesize full-resolution, stereo frames in a head-mounted display, updating the rendered perspective in real-time based on the viewer’s head position and orientation. Finally, we propose a theoretical model for understanding the disocclusion behavior in depth-based novel view synthesis and analyze the impact of the choice of intermediate representation and camera geometry on the synthesized views in terms of quantitative image quality metrics and the occurrence of disocclusion holes.
Bio: Jayant Thatte is a graduating Ph.D. student in Electrical Engineering, advised by Prof. Bernd Girod. His interest areas include computer vision, virtual reality, and machine learning. His Ph.D. research focuses on enabling real-time rendering of head-motion parallax for real-world virtual reality content. Additionally, he also serves as a teaching assistant for EE 368: Digital Image Processing. He is expected to graduate at the end of December and is actively looking for a full-time job. Personal webpage: https://web.stanford.edu/~jayantt/
