Posters:
Note: This is a partial list of posters -check back to see updates
Deep Sensor Fusion for 3D Imaging by Mark Nishimura, David Lindell, Matthew O’Toole, Gordon Wetzstein
Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning by Chi-Sing Ho, Neal Jean, Catherine Hogan, Lena Blackmon, Stefanie Jeffrey, Mark Holodniy, Niaz Banaei, Amr Saleh, Stefano Ermon, Jennifer Dionne
A Time-of-Flight Imaging System Based on a CMOS Image Sensor by Okan Atalar, Raphael Van Laer, Christopher J. Sarabalis, Amir H. Safavi-Naeini, Amin Arbabian
Mapping Histological Slice Sequence to the Allen Mouse Brain Atlas Without 3D Reconstruction by Jing Xiong, Jing Ren, Liqun Luo, Mark Horowitz
Registration of Surgical Microscope Images with a CT Scan by Lars Jebe and Bernd Girod
A convex 3D deconvolution algorithm for low photon count fluorescence imaging by Hayato Ikoma, Michael Broxton, Takamasa Kudo, Gordon Wetzstein
Non-Line-of-Sight Imaging by David B. Lindell, Matthew O’Toole, Vladlen Koltun, Gordon Wetzstein
Spherical-aberration-assisted extended depth-of-field (SPED) light sheet microscopy by Raju Tomer, Matthew Lovett-Barron, Isaac Kauvar, Aaron Andalman, Vanessa Burns, Sethuraman Sankaran, Logan Grosenick, Michael Broxton, Samuel Yang, Karl Deisseroth
Accommodation-Invariant Near-Eye Displays by Robert Konrad, Nitish Padmanaban, Keenan Molner, Emily A. Cooper, Gordon Wetzstein
Ray-tracing 3D spectral scenes through human optics by Trisha Lian, Kevin J. MacKenzie, Brian Wandell
Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification by Julie Chang, Vincent Sitzmann, Xiong Dun, Wolfgang Heidrich, and Gordon Wetzstein
An Extended Depth-of-Field Volumetric Near-Eye Augmented Reality Display by Kishore Rathinave, Hanpeng Wang, Alex Blate, Henry Fuchs
A system for generating complex physically accurate sensor images for automotive applications by Zhenyi Liu, Minghao Shen, Jiaqi Zhang, Shuangting Liu, Henryk Blasinski, Trisha Lian, and Brian Wandell
The Deep Computational Camera: domain-specific imaging with end-to-end optimized vision pipelines by Vincent Sitzmann, Steven Diamond, Evan Peng, Xiong Dun, Wolfgang Heidrich, Stephen Boyd, Felix Heide, Gordon Wetzstein
Full-Field Interferometric Imaging of Action Potentials by Kevin C Boyle, Tong Ling, Georges Goetz, Felix S Alfonso, Tiffany Huang, Daniel Palanker
MF-PAT: Accelerated Precomputed Acoustic Transfer Using Multi-Frequency Packing by Jui-Hsien Wang and Doug L. James
Short-Wave Infra Red Otoscopy for diagnosis of Otitis Media by S.P.Singh, David Zarabanda and Tulio Valdez
Stacked OmniStereo Representation for Virtual Reality with Head-Motion Parallax by Jayant Thatte and Bernd Girod
Training the linear, local, learned (L3) pipeline for imaging in low light conditions by Zheng Lyu and Brian Wandell
Metamaterial design based on optimization and machine learning by Jiaqi Jiang, David Sell, Thaibao Phan, Evan Wang, Jianji Yang, Jonathan A. Fan
Deep learning to automatically validate diffusion MRI acquisition parameters by Fabian Reith, L. Michael Perry, Garikoitz Lerma-Usabiaga
Opt 2: A DSL for Non-linear Least Squares Optimization on GPUs by Michael Mara
Gaze-Contingent Eyeglasses for Presbyopes by Nitish Padmanaban, Robert Konrad, Gordon Wetzstein
Abstracts
Title: Deep Sensor Fusion for 3D Imaging
Authors: Mark Nishimura, David Lindell, Matthew O’Toole, Gordon Wetzstein
Abstract: Sensors which capture 3D scene information provide useful data for tasks in vehicle navigation, gesture recognition, human pose estimation, and geometric reconstruction. Active illumination time-of-flight sensors in particular have become widely used to estimate a 3D representation of a scene. However, current systems are fundamentally limited by a tradeoff between range, acquisition speed, and resolution. We propose two data-driven methods for photon-efficient 3D imaging which alleviate this tradeoff by leveraging sensor fusion and deep learning to estimate a dense depth map, the first drastically reducing signal requirements, and the second eliminating the need for scanning. Our first approach robustly recovers depth at extreme low-signal levels by scanning and collecting measurements over a 3D spatio-temporal volume and using a grayscale image to improve accuracy and resolution. Our second approach uses only a single pixel depth sensor and RGB image, estimating depth at a higher resolution without scanning. We demonstrate the efficacy of our approaches on both simulated and real world data for the former, and on simulated data for the latter.
Bio: Mark Nishimura is a first-year Ph.D student in Electrical Engineering. He completed his B.S. and M.S. degrees in Electrical Engineering at Stanford University and is currently interested in computer vision and optimization.
Title: Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning
Authors: Chi-Sing Ho, Neal Jean, Catherine Hogan, Lena Blackmon, Stefanie Jeffrey, Mark Holodniy, Niaz Banaei, Amr Saleh, Stefano Ermon, Jennifer Dionne
Abstract: Rapid identification of bacteria is essential to prevent the spread of infectious disease, help combat antimicrobial resistance, and improve patient outcomes. Raman optical spectroscopy promises to combine bacterial detection, identification, and antibiotic susceptibility testing in a single step. However, achieving clinically relevant speeds and accuracies remains challenging due to the weak Raman signal from bacterial cells and the large number of bacterial species and phenotypes. By amassing the largest known dataset of bacterial Raman spectra, we are able to apply state-of-the-art deep learning approaches to identify 30 of the most common bacterial pathogens from noisy Raman spectra, achieving antibiotic treatment identification accuracies of 99.0%. Results from initial clinical validation are promising: using just 10 bacterial spectra from each of 25 isolates, we achieve 99.0% species identification accuracy. Our combined Raman-deep learning system represents an important proof-of-concept for rapid, culture-free identification of bacterial isolates and antibiotic resistance and could be readily extended for diagnostics on blood, urine, and sputum.
Bio: Shing Shing Ho is a 6th year Applied Physics PhD student. Her research interests include applications of deep learning to high throughput imaging and spectroscopy, development of automated data acquisition systems, and nanomaterials and optical systems for clinical diagnostics and therapeutics.
Title: A Time-of-Flight Imaging System Based on a CMOS Image Sensor
Authors: Okan Atalar, Raphael Van Laer, Christopher J. Sarabalis, Amir H. Safavi-Naeini, Amin Arbabian
Abstract: Time-of-flight (ToF) imaging is a key enabling technology for many fields, including machine vision, robotics, tracking, and autonomous vehicles. Standard image sensors can form high resolution images of a scene, but have limited depth information. State of the art ToF cameras, on the other hand, can generate accurate depth images but are expensive and have limited spatial resolution due to custom designed pixels. In this work, we integrate a “free-space optical mixer” to a standard CMOS image sensor to convert it into a ToF camera. We illuminate a scene with megahertz level amplitude modulated light and downconvert the reflected light into the temporal bandwidth of the CMOS image sensor using the free-space optical mixer. The CMOS image sensor detects the hertz level beat tones. Using the measured phases for the beat tones with signal processing techniques, ToF imaging with a standard CMOS image sensor is realized.
Bio: Okan Atalar is a 3rd year PhD candidate in the Electrical Engineering Department at Stanford University, advised by Amir H. Safavi-Naeini and Amin Arbabian. His research interests include time-of-flight imaging and optical interferometry.
Title: Mapping Histological Slice Sequence to the Allen Mouse Brain Atlas Without 3D Reconstruction
Authors: Jing Xiong, Jing Ren, Liqun Luo, Mark Horowitz
Abstract: Histological brain slices are widely used in neuroscience to study the anatomical organization of neural circuits. Systematic and accurate comparisons of anatomical data from multiple brains, especially from different studies, can benefit tremendously from registering histological slices onto a common reference atlas. Most existing methods rely on an initial reconstruction of the volume before registering it to a reference atlas. Because these slices are prone to distortion during the sectioning process and often sectioned with non-standard angles, reconstruction is challenging and often inaccurate. Here we describe a framework that maps each slice to its corresponding plane in the Allen Mouse Brain Atlas to build a plane-wise mapping and then perform 2D nonrigid registration to build a pixel-wise mapping. We use the L2 norm of the Histogram of Oriented Gradients (HOG) of two patches as the similarity metric for both steps, and a Markov Random Field formulation that incorporates tissue coherency to compute the nonrigid registration. To fix significantly distorted regions that are misshaped or much smaller than the control grids, we trained a context-aggregation network to segment and warp them to their corresponding regions with thin plate spline. We have shown that our method generates results comparable to an expert neuroscientist and is significantly better than reconstruction-first approaches.
Bio: Jing Xiong is her final year of the PhD program in electrical engineering, working on brain mapping algorithms with Professor Mark Horowitz. She works closely and is co-advised by Professor Liqun Luo from the biology department. She is interested in applying computer vision/image processing techniques to help neuroscientists obtain more precise information from their imagery data.
Title: Registration of Surgical Microscope Images with a CT Scan
Authors: Lars Jebe, Bernd Girod
Abstract: The ARRISCOPE is one of the first digital stereo microscopes intended for surgery. There is no optical path to the surgeon’s eyes, instead the surgeon looks at two screens. This enables the incorporation of augmented reality into the microscope. Information from a preoperative CT scan can be shown to the surgeon, for example by displaying a nerve or a tumor that is hidden underneath the surface. A first step to accurately displaying such information is the registration of the camera position relative to the patient. Therefore, the stereo images from the camera have to be aligned with the CT scan. Traditional methods such as the ICP algorithm are not able to perform this task, due to the large complexity of the CT scan and the limited amount of data from the microscope. Our registration procedure includes depth estimation, design and matching of local shape descriptors, and a geometric verification to find and refine the alignment.
Bio: Lars Jebe is a second year Master’s student in Electrical Engineering with research interests in Computational Imaging and Computer Vision. He received a Bachelor’s degree in Information Theory and Communication Technology from RWTH Aachen University, Germany in 2017.
Title: A convex 3D deconvolution algorithm for low photon count fluorescence imaging
Authors: Hayato Ikoma, Michael Broxton, Takamasa Kudo, Gordon Wetzstein
Abstract: Deconvolution is widely used to improve the contrast and clarity of a 3D focal stack collected using a fluorescence microscope. But despite being extensively studied, deconvolution algorithms can introduce reconstruction artifacts when their underlying noise models or priors are violated, such as when imaging biological specimens at extremely low light levels. In this paper we propose a deconvolution method specifically designed for 3D fluorescence imaging of biological samples in the low-light regime. Our method utilizes a mixed Poisson-Gaussian model of photon shot noise and camera read noise, which are both present in low light imaging. We formulate a convex loss function and solve the resulting optimization problem using the alternating direction method of multipliers algorithm. Among several possible regularization strategies, we show that a Hessian-based regularizer is most effective for describing locally smooth features present in biological specimens. Our algorithm also estimates noise parameters on-the-fly, thereby eliminating a manual calibration step required by most deconvolution software. We demonstrate our algorithm on simulated images and experimentally-captured images with peak intensities of tens of photoelectrons per voxel. We also demonstrate its performance for live cell imaging, showing its applicability as a tool for biological research.
Bio: I am a Ph.D. student at Department of Electrical Engineering, Stanford University and a member of Stanford Computational Imaging Group. For my Ph.D. research, I am focusing on the development of computational imaging techniques for fluorescence optical microscopy and broadly interested in signal processing, machine learning and optimization. I am currently working on the application of convolutional neural network for optical microscopy. I also serve as a teaching assistant for EE267: Virtual Reality.
Title: A system for generating complex physically accurate sensor images for automotive applications
Authors: Zhenyi Liu, Minghao Shen, Jiaqi Zhang, Shuangting Liu, Henryk Blasinski, Trisha Lian, and Brian Wandell
Abstract: We describe an open-source simulator that automates the creation of sensor irradiance and sensor images of typical automotive scenes in urban settings. The methods specify scene parameters (e.g., scene type, road type, traffic density, weather conditions) to assemble random scenes from graphics assets stored in a database. The sensor irradiance is generated using quantitative computer graphics methods, and the images are created using image systems sensor simulation. The sensor images used to train and evaluate neural networks.
Bio: Zhenyi Liu is a 3rd year Ph.D from Jilin University and a visiting student in Prof. Brian Wandell’s lab. His research involves developing methods of generating physically accurate virtual scenes that are used in simulations to evaluate camera and lidar sensors for automotive applications.
Title: Non-Line-of-Sight Imaging
Authors: David B. Lindell, Matthew O’Toole, Vladlen Koltun, Gordon Wetzstein
Abstract: Non-line-of-sight (NLOS) imaging enables unprecedented capabilities in a wide range of applications, including robotic and machine vision, remote sensing, autonomous vehicle navigation, and medical imaging. We present our recent approach to solving this challenging problem using optical and acoustic sensing modalities. In the optical regime, we reconstruct the 3D geometry of a hidden scene captured from “around the corner” using highly sensitive time-resolved photodetectors and ultra-fast pulsed lasers. Our reconstruction algorithm, based on the light-cone transform, substantially improves upon the memory and computational requirements of other methods. Furthermore, we present an interactive demo of our NLOS imaging system. We also introduce acoustic NLOS imaging, which is orders of magnitude less expensive than most optical systems and captures hidden 3D geometry at longer ranges with shorter acquisition times compared to state-of-the-art optical methods. Inspired by hardware setups used in radar and algorithmic approaches to model and invert wave-based image formation models developed in the seismic imaging community, we demonstrate a new approach to seeing around corners.
Bio: David is a 3rd-year PhD student in the Computational Imaging Lab advised by Gordon Wetzstein. His research involves developing computational methods and hardware systems for time-of-flight 3D imaging.
Title: Spherical-aberration-assisted extended depth-of-field (SPED) light sheet microscopy
Authors: Raju Tomer, Matthew Lovett-Barron, Isaac Kauvar, Aaron Andalman, Vanessa Burns, Sethuraman Sankaran, Logan Grosenick, Michael Broxton, Samuel Yang, Karl Deisseroth
Abstract: The goal of understanding living nervous systems has driven interest in high-speed and large field-of-view volumetric imaging at cellular resolution. Light sheet microscopy approaches have emerged for cellular-resolution functional brain imaging in small organisms such as larval zebrafish, but remain fundamentally limited in speed. Here, we have developed SPED light sheet microscopy, which combines large volumetric field-of-view via an extended depth
of field (EDOF) with the optical sectioning of light sheet microscopy, thereby eliminating the need to physically scan detection objectives for volumetric imaging. SPED is thus not limited by the speed of piezo or tunable-lens based focal scanning, one of the major roadblocks to higher imaging speeds. We perform simulations and experiments exploring the design space of spherical aberration based EDOF, and we demonstrate capabilities of SPED microscopy for fast cellular resolution volumetric imaging of entire zebrafish nervous systems.
Bio: Isaac Kauvar is a PhD candidate in Electrical Engineering, co-advised by Dr. Gordon Wetzstein and Dr. Karl Deisseroth. His research focuses on applications of computational optics to neuroscience.
Title: Accommodation-Invariant Near-Eye Displays
Authors: Robert Konrad, Nitish Padmanaban, Keenan Molner, Emily A. Cooper, Gordon Wetzstein
Abstract: Although emerging virtual and augmented reality (VR/AR) systems can produce highly immersive experiences, they can also cause visual discomfort, eyestrain, and nausea. One of the sources of these symptoms is a mismatch between vergence and focus cues. In current VR/AR near-eye displays, a stereoscopic image pair drives the vergence state of the human visual system to arbitrary distances, but the accommodation, or focus, state of the eyes is optically driven towards a fixed distance. In this work, we introduce a new display technology, dubbed accommodation-invariant (AI) near-eye displays, to improve the consistency of depth cues in near-eye displays. Rather than producing correct focus cues, AI displays are optically engineered to produce visual stimuli that are invariant to the accommodation state of the eye. The accommodation system can then be driven by stereoscopic cues, and the mismatch between vergence and accommodation state of the eyes is significantly reduced. We validate the principle of operation of AI displays using a prototype display that allows for the accommodation state of users to be measured while they view visual stimuli using multiple different display modes.
Bio: Robert is a 5th year PhD candidate in the Electrical Engineering Department at Stanford University, advised by Professor Gordon Wetzstein as part of the Stanford Computational Imaging Lab. His research interests lie at the intersection of computational displays and human physiology with a specific focus on virtual and augmented reality systems. He has recently worked on relieving vergence-accommodation and visual-vestibular conflicts present in current VR and AR displays, as well a computationally efficient cinematic VR capture system. He received his Bachelor’s Degree from the ECE department at the University of Toronto in 2014, and his Master’s Degree from the EE Department at Stanford University in 2016.
Title: Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification
Authors: Julie Chang, Vincent Sitzmann, Xiong Dun, Wolfgang Heidrich, and Gordon Wetzstein
Abstract: Increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.
Bio: Julie Chang is a 5th year PhD student advised by Gordon Wetzstein. Her research projects have explored different applications of diffraction-aware computational imaging including light field microscopy and photography, imaging through scattering media, and optical neural networks.
Title: An Extended Depth-of-Field Volumetric Near-Eye Augmented Reality Display
Authors: Kishore Rathinave, Hanpeng Wang, Alex Blate, Henry Fuchs
Abstract: This poster presents an optical design and a rendering pipeline for a full-color volumetric near-eye display which simultaneously presents imagery with near-accurate per-pixel focus across an extended volume ranging from 15cm (6.7 diopters) to 4M (0.25 diopters), allowing the viewer to accommodate freely across this entire depth range. This is achieved using a focus-tunable lens that continuously sweeps a sequence of 280 synchronized binary images from a high-speed, Digital Micromirror Device (DMD) projector and a high-speed, high dynamic range (HDR) light source that illuminates the DMD images with a distinct color and brightness at each binary frame. Our rendering pipeline converts 3-D scene information into a 2-D surface of color voxels, which are decomposed into 280 binary images in a voxel-oriented manner, such that 280 distinct depth positions for full-color voxels can be displayed, refreshed at 60 Hz.
Bio: Kishore Rathinavel is a visiting student at the Stanford Computational Imaging Lab, advised by Prof. Gordon Wetzstein. Kishore is pursuing a Ph.D. in Computer Science at UNC-Chapel Hill, advised by Prof. Henry Fuchs. His Ph.D. research is focused on Near-Eye Display technologies. He received a B.Tech. in Electrical Engineering (minor in Computer Science) from IIT Gandhinagar, India, in 2012 and an M.S. in Computer Science from UNC-Chapel Hill in 2016. In 2012-13, he was an Associate Engineer at Ricoh Innovations, Bangalore, India.
Title: The Deep Computational Camera: domain-specific imaging with end-to-end optimized vision pipelines
Authors: Vincent Sitzmann, Steven Diamond, Evan Peng, Xiong Dun, Wolfgang Heidrich, Stephen Boyd, Felix Heide, Gordon Wetzstein
Abstract: Soon, most of the world’s imagery will be consumed by machines for purposes of computer vision (CV). Yet, through the decades, our image acquisition and post-processing techniques have been optimized exclusively for images that are nice to look at – not easily interpreted by a CV system. With the Deep Computational Camera, we want to optimize the full imaging pipeline end-to-end for true domain-specific imaging. We introduce novel methods to optimize camera optics, image reconstruction, and neural networks jointly in a stochastic gradient descent framework.
Bio: Vincent is a third-year Ph.D. student in the Stanford Computational Imaging Laboratory, advised by Prof. Gordon Wetzstein. His research interests lie at the intersection of machine-learning driven Computer Vision and Computational Imaging, with brief stints in VR and Human Perception.
Title: Ray-tracing 3D spectral scenes through human optics
Authors: Trisha Lian, Kevin J. MacKenzie, Brian Wandell
Abstract: Display technology design benefits from a quantitative understanding of how parameters of novel displays impact the retinal image. Vision scientists have developed many precise computations and facts that characterize critical steps in vision, particularly at the first stages of light encoding. ISETBIO is an open-source implementation that aims to provide these computations. The initial implementation modeled image formation for distant or planar scenes. Here, we extend ISETBIO by using computer graphics and ray-tracing to model how spectral, three-dimensional scenes are transformed by human optics to the retinal irradiance. The extended software allows the user to specify a model of the physiological optics that can be used to ray trace from the scene to the retina, accounting for the three-dimensional scene as well as optical factors such as chromatic aberration, accommodation, pupil size, and diffraction. We describe and test the implementation for the Navarro eye model, and quantify several features of the physiological optics that define important effects of three-dimensional image formation. Potential applications of these methods include understanding the impacts of occlusion, binocular vision, and 3D displays on the retinal image.
Bio: Trisha is an Electrical Engineering PhD student at Stanford University. Before Stanford, she received her bachelor’s in Biomedical Engineering from Duke University. She is currently advised by Professor Brian Wandell and works on interdisciplinary topics that involve image systems simulations. These range from novel camera designs to simulations of the human visual system.
Title: Full-Field Interferometric Imaging of Action Potentials
Authors: Kevin C Boyle, Tong Ling, Georges Goetz, Felix S Alfonso, Tiffany Huang, Daniel Palanker
Abstract: We demonstrate full-field imaging of the action potentials based on quantitative phase mapping of the associated movement of the cell membrane. Deformations of up to 3nm (0.9 mrad) with a rise time of 4ms in spiking HEK cells match the electrical waveforms. Since the shot noise limit of the camera (~2 mrad/pix) precludes detection of the action potential in a single frame, for all-optical spike detection, images are acquired at 50 kHz, and 50 frames are binned into 1 ms steps to achieve a sensitivity of 0.3 mrad in a single pixel. Using spatial averaging over a cell, individual spikes can be detected by matching the previously extracted template of the action potential with the optical recording. This allows all-optical full-field imaging of the propagating action potentials without exogeneous labels or electrodes.
Bio: Kevin is a PhD Candidate in Electrical Engineering studying the mechanical deformation that accompanies the action potential in neurons. Dr. Ling is a post-doctoral fellow working on optophysiology, laser-tissue interactions and applications of optical interferometry.
Title: MF-PAT: Accelerated Precomputed Acoustic Transfer Using Multi-Frequency Packing
Authors: Jui-Hsien Wang and Doug L. James
Abstract: We present a new method to accelerate the precomputed acoustic transfer algorithm (PAT). PAT enables real-time synthesis of convincing rigid-body sounds; however, the precomputation step can take a long time due to its single-frequency Helmholtz solve. We propose an efficient transfer estimation algorithm that leads to efficient precomputation using multi-frequency packing. Our GPU implementation provides speedup of two orders of magnitudes compared to the traditional method.
Bio: Jui-Hsien is a fourth year PhD student in the Institute of Computational and Mathematical Engineering at Stanford. He is advised by Doug James. His research interest primarily lies in physical simulations for computer graphics and robotics.
Title: Short-Wave Infra Red Otoscopy for diagnosis of Otitis Media
Authors: S.P.Singh, David Zarabanda and Tulio Valdez
Abstract: Otitis media is one of the most common reasons for pediatrician visits, antibiotic prescription, and surgery in the pediatric population. Visible light pneumatic-otoscopy is considered the best currently available diagnostic tool for otitis media. However, it has various limitations e.g. the disposable speculum cannot create an adequate seal against the external auditory canal to obtain tympanic membrane movement. Also, lack of training for effective pneumatic-otoscopy for most clinicians is another factor. To overcome these limitations, we have recently developed an otoscope sensitive to shortwave infrared (SWIR) wavelengths of light. A SWIR otoscope could help identify middle-ear-effusions based on the strong light absorption by ear fluid. Due to a longer wavelength, light can penetrate deeper through tissue, enabling a better view behind the tympanic membrane. Here we present our preliminary findings on the feasibility of using video rate SWIR imaging in a pediatric population.
Bio: S.P.Singh is a postdoctoral fellow at Valdez Lab in OHNS department at Stanford. His expertise includes applying different spectroscopic approaches for biomedical applications. David Zarabanda is a postdoctoral fellow in the same lab and his expertise lies in developing animal models for allergy, inflammation. Tulio Valdez is an associate professor in OHNS department. He has long-term experience in utilizing SWIR light for different biomedical applications.
Title: Stacked OmniStereo Representation for Virtual Reality with Head-Motion Parallax
Authors: Jayant Thatte and Bernd Girod
Abstract: The quality of experience in virtual reality is a function of the fidelity with which a rendering system reproduces the wide array of visual cues that are employed by our brains to perceive reality. In this work, we limit our attention to two prominent depth cues, namely head-motion parallax and binocular stereopsis. We summarize the impact of the presence or absence of these cues on two specific measures of quality: overall viewer preference and accuracy of size perception, which is an aspect of realism. Additionally, we also present a novel data representation called Stacked OmniStereo and an end-to-end system for rendering virtual reality with motion parallax. We present an algorithm to construct the proposed representation from a stationary camera rig system and a real-time renderer that can synthesize stereo novel views in response to the viewer’s 6 degrees-of-freedom head-motion. Our proposed representation comprises only 4 texture-plus-depth panoramic images and therefore is significantly compact compared to light fields and the synthesized novel views yield better PSNR and SSIM results compared to competing methods when evaluated against computer-generated ground truth images. We believe that Stacked OmniStereo puts forth an elegant method of rendering head-motion parallax, which plays a critical role in the overall quality of experience in virtual reality.
Bio: Jayant Thatte received his B.Tech and M.Tech in electrical engineering from Indian Institute of Technology Madras (2014) along with Philips India Award for best academic record. He is currently a Ph.D. candidate in electrical engineering at Stanford University. His work is focused on the development of image processing algorithms and systems that provide a more natural and comfortable cinematic virtual reality experience.
Title: Training the linear, local, learned (L3) pipeline for imaging in low light conditions
Authors: Zheng Lyu and Brian Wandell
Abstract: The Local, Linear and Learned (L3) method is a machine-learning approach to generating image processing pipelines [1]. The method conceives of the image processing pipeline as selecting and then applying one of a large collection of filters to patches surrounding each pixel in the captured data. The filter outputs are the rendered image. The filters are learned from large collections of data derived using image system simulations. We used the L3 to learn the filters for reducing the noise in images acquired under low illumination conditions. I will explain the methods, show the filters that were learned, and compare the L3 rendering with conventional noise reduction methods. [1] Jiang, Haomiao, et al. “Learning the image processing pipeline.” IEEE Transactions on Image Processing 26.10 (2017): 5032-5042.
Bio: Zheng Lyu is a PhD student in EE department at Stanford University. His interest includes image process, machine learning, deep learning. Zheng received his BEng degree from Tsinghua University.
Title: Metamaterial design based on optimization and machine learning
Authors: Jiaqi Jiang, David Sell, Thaibao Phan, Evan Wang, Jianji Yang, Jonathan A. Fan
Abstract: Metasurfaces are ultra-thin optical elements that have great promise for constructing lightweight and compact optical systems. For their practical implementation, it is imperative to maximize metasurface efficiency. Topology optimization provides a pathway to pushing the limits of metasurface efficiency, but these methods have been limited to the design of microscale devices due to the extensive computational resources required. We introduce new strategies to optimize large-area metasurfaces in a computationally-efficient manner.
Bio: Thaibao Phan is a 3rd year Ph.D. student in Electrical Engineering. His research interests include tunable optical systems and compact radio-frequency antenna systems.
Title: Deep learning to automatically validate diffusion MRI acquisition parameters
Authors: Fabian Reith, L. Michael Perry, Garikoitz Lerma-Usabiaga
Abstract: Diffusion weighted imaging (DWI) is an important magnetic resonance imaging (MRI) modality for assessing the health and structures in the white matter pathways of the human brain. The parameters and the raw data (3D images) obtained when acquiring DWI scans are encoded through separate software pathways. In some cases, the critical parameters (e.g., the diffusion directions) and images are stored incorrectly. We developed a deep-learning algorithm based on a fully connected artificial neural network to automatically detect this mismatch. We used a large set of diffusion imaging data acquired at Stanford’s Center for Neurobiological Imaging and stored in a Flywheel.io database to train the network. The data had been validated by scientists over the last seven years, and the incorrect data were created by purposefully by introducing label errors. The network assesses the likelihood of an error and automatically corrects the parameters and informs the user. Tested on more than 400 MRI data sets, network performance is essentially error-free.
Bio: Fabian Reith is a graduate student at the Technical University of Munich and visiting the EE department at Stanford University where he is working with Prof. Brian Wandell. His interests include computer vision, machine learning and deep learning.
Title: Opt 2: A DSL for Non-linear Least Squares Optimization on GPUs
Authors: Michael Mara
Abstract: Many graphics and vision problems are naturally expressed as optimizations with either linear or non-linear least squares objective functions over visual data, such as images and meshes. The mathematical descriptions of these functions are extremely concise, but their implementation in real code is tedious, especially when optimized for real-time performance in interactive applications. Opt is a new language in which a user simply writes high-level energy functions, and a compiler automatically generates state-of-the-art GPU optimization kernels. The end result is a system in which real-world energy functions in graphics and vision applications are expressible in tens of lines of code. They compile directly into highly optimized GPU solver implementations with performance competitive with the best published hand-tuned, application-specific GPU solvers, and 1–2 orders of magnitude beyond a general-purpose auto-generated solver. The original version of Opt was restricted to maximal performance on certain classes of NLLS problems. By simplifying the energy language introducing a scheduling co-language that parameterizes a space of structural data-parallel code transforms (and an auto scheduler) we obtain state-of-the-art performance on a massive space of problems, from deconvolution, to blend shape fitting, to bundle adjustment.
Bio: Michael Mara is a computer graphics researcher whose work straddles several subfields, and focuses on order-of-magnitude performance improvements to bring interesting offline techniques into real-time. He is currently a PhD candidate at Stanford, advised by Pat Hanrahan. His previous work includes real-time rendering research at NVIDIA Research, Oculus Research, and Facebook Reality Labs.
Title: Autofocals: Gaze-Contingent Eyeglasses for Presbyopes
Authors: Nitish Padmanaban, Robert Konrad, Gordon Wetzstein
Abstract: As humans age, they gradually lose the ability to accommodate, or refocus, to near distances due to the stiffening of the crystalline lens. Known as presbyopia, this condition affects nearly 20% of people worldwide. Traditional corrections for presbyopia use fixed focal elements, inherently impairing field of view or stereo vision to regain a greater range of sharp vision. Moreover, none of these operate the same way as the natural refocusing mechanism. We design and build a new presbyopia correction, autofocals, to externally mimic the natural accommodation response, combining eye tracker and depth sensor data to automatically drive focus-tunable lenses. We evaluate our autofocal prototype with a total of 56 users. In the first study, 19 users were evaluated across a set of visual and task performance metrics: visual acuity, contrast sensitivity, and a refocusing task. Autofocals exhibit better visual acuity on average across a range of focusing distances when compared to monovision and progressive lenses, while maintaining similar contrast sensitivity. On the refocusing task, users are faster when using autofocals, and compared to progressives, also significantly more accurate. In a separate qualitative study, a clear majority of 23 of 37 users rank autofocals as the best correction when comparing autofocals against their own correction in terms of ease of refocusing to multiple distances. Digital eyeglasses are an emerging, yet unproven technology that could impact the lives of millions; our work is the first to demonstrate their superiority over current forms of presbyopia correction.
