Alperovich, Anna
E-Mail-Adresse
ORCID
Geburtsdatum
Berufsbeschreibung
Nachname
Vorname
Name
Suchergebnisse Publikationen
Synthetic data generation for optical flow evaluation in the neurosurgical domain
Towards computer-assisted neurosurgery, scene understanding algorithms for microscope video data are required. Previous work utilizes optical flow to extract spatiotemporal context from neurosurgical video sequences. However, to select an appropriate optical flow method, we need to analyze which algorithm yields the highest accuracy for the neurosurgical domain. Currently, there are no benchmark datasets available for neurosurgery. In our work, we present an approach to generate synthetic data for optical flow evaluation on the neurosurgical domain. We simulate image sequences and thereby take into account domainspecific visual conditions such as surgical instrument motion. Then, we evaluate two optical flow algorithms, Farneback and PWC-Net, on our synthetic data. Qualitative and quantitative assessments confirm that our data can be used to evaluate optical flow for the neurosurgical domain. Future work will concentrate on extending the method by modeling additional effects in neurosurgery such as elastic background motion.
An Epipolar Volume Autoencoder With Adversarial Loss for Deep Light Field Super-Resolution
When capturing a light field of a scene, one typically faces a trade-off between more spatial or more angular resolution. Fortunately, light fields are also a rich source of information for solving the problem of super-resolution. Contrary to single image approaches, where high-frequency content has to be hallucinated to be the most likely source of the downscaled version, sub-aperture views from the light field can help with an actual reconstruction of those details that have been removed by downsampling. In this paper, we propose a three-dimensional generative adversarial autoencoder network to recover the high-resolution light field from a low-resolution light field with a sparse set of viewpoints. We require only three views along both horizontal and vertical axis to increase angular resolution by a factor of three while at the same time increasing spatial resolution by a factor of either two or four in each direction, respectively.
Shadow and Specularity Priors for Intrinsic Light Field Decomposition
In this work, we focus on the problem of intrinsic scene decomposition in light fields. Our main contribution is a novel prior to cope with cast shadows and inter-reflections. In contrast to other approaches which model inter-reflection based only on geometry, we model indirect shading by combining geometric and color information. We compute a shadow confidence measure for the light field and use it in the regularization constraints. Another contribution is an improved specularity estimation by using color information from sub-aperture views. The new priors are embedded in a recent framework to decompose the input light field into albedo, shading, and specularity. We arrive at a variational model where we regularize albedo and the two shading components on epipolar plane images, encouraging them to be consistent across all sub-aperture views. Our method is evaluated on ground truth synthetic datasets and real world light fields. We outperform both state-of-the art approaches for RGB+D images and recent methods proposed for light fields.
Accurate Depth and Normal Maps from Occlusion-Aware Focal Stack Symmetry
We introduce a novel approach to jointly estimate consistent depth and normal maps from 4D light fields, with two main contributions. First, we build a cost volume from focal stack symmetry. However, in contrast to previous approaches, we introduce partial focal stacks in order to be able to robustly deal with occlusions. This idea already yields significanly better disparity maps. Second, even recent sublabel-accurate methods for multi-label optimization recover only a piecewise flat disparity map from the cost volume, with normals pointing mostly towards the image plane. This renders normal maps recovered from these approaches unsuitable for potential subsequent applications. We therefore propose regularization with a novel prior linking depth to normals, and imposing smoothness of the resulting normal field. We then jointly optimize over depth and normals to achieve estimates for both which surpass previous work in accuracy on a recent benchmark.
Digital core repository coupled with machine learning as a tool to classify and assess petrophysical rock properties
To make efficient use of image-based rock physics workflow, it is necessary to optimize different criteria, among which: quantity, representativeness, size and resolution. Advances in artificial intelligence give insights of databases potential. Deep learning methods not only enable to classify rock images, but could also help to estimate their petrophysical properties. In this study we prepare a set of thousands high-resolution 3D images captured in a set of four reservoir rock samples as a base for learning and training. The Voxilon software computes numerical petrophysical analysis. We identify different descriptors directly from 3D images used as inputs. We use convolutional neural network modelling with supervised training using TensorFlow framework. Using approximately fifteen thousand 2D images to drive the classification network, the test on thousand unseen images shows any error of rock type misclassification. The porosity trend provides good fit between digital benchmark datasets and machine learning tests. In a few minutes, database screening classifies carbonates and sandstones images and associates the porosity values and distribution. This work aims at conveying the potential of deep learning method in reservoir characterization to petroleum research, to illustrate how a smart image-based rock physics database at industrial scale can swiftly give access to rock properties.
Intrinsic Light Field Decomposition and Disparity Estimation with Deep Encoder-Decoder Network
We present an encoder-decoder deep neural network that solves non-Lambertian intrinsic light field decomposition, where we recover all three intrinsic components: albedo, shading, and specularity. We learn a sparse set of features from 3D epipolar volumes and use them in separate decoder pathways to reconstruct intrinsic light fields. While being trained on synthetic data generated with Blender, our model still generalizes to real world examples captured with a Lytro Illum plenoptic camera. The proposed method outperforms state-of-the-art approaches for single images and achieves competitive accuracy with recent modeling methods for light fields.
A Taxonomy and Evaluation of Dense Light Field Depth Estimation Algorithms
This paper presents the results of the depth estimation challenge for dense light fields, which took place at the second workshop on Light Fields for Computer Vision (LF4CV) in conjunction with CVPR 2017. The challenge consisted of submission to a recent benchmark [7], which allows a thorough performance analysis. While individual results are readily available on the benchmark web page http://www.lightfield-analysis.net, we take this opportunity to give a detailed overview of the current participants. Based on the algorithms submitted to our challenge, we develop a taxonomy of light field disparity estimation algorithms and give a report on the current state-of-the-art. In addition, we include more comparative metrics, and discuss the relative strengths and weaknesses of the algorithms. Thus, we obtain a snapshot of where light field algorithm development stands at the moment and identify aspects with potential for further improvement.
Variational and Deep Learning Approaches for Intrinsic Light Field Decomposition
Intrinsic image decomposition aims to separate an illumination invariant reflectance image from an input color image, which is still one of the fundamental problems in computer vision. This decomposition is widely used in photo and material editing, image segmentation and shape estimation tasks. According to the dichromatic reflection model, the light reflected from a scene point has two independent components: light reflected from the surface body and light at the interface. Body reflection is known as the diffuse component and it is independent of viewing direction, while interface reflection is known as the specular component and it is view-dependent. Most intrinsic image algorithms are designed for Lambertian scenes, with only diffuse reflection. However, their performance decreases if a scene contains specularity. In the real world, there are few scenes with only Lambertian objects. Instead, they have specular surfaces, which makes the decomposition problem harder due to the complicated nature of specular reflection. This thesis focuses on intrinsic light field decomposition, where we formulate and solve the problem with respect to three variables: albedo, shading, and specularity. Thus, we can deal with non-Lambertian scenes. We use a 4D light field, which is a collection of images sampled on a regular grid, instead of a single image. Rich information inherited from the light field allows us to distinguish between diffuse and specular reflection, and also allows us to robustly recover the intrinsic components. We tackle the problem with variational and deep learning approaches, compare their performance, and discuss the strengths and weaknesses of both techniques. In the variational method, we introduce priors for the intrinsic components and we solve an energy minimization problem with convex optimization. Because geometrical information plays an important role in the appearance and behavior of intrinsic components, we develop a disparity estimation method, where we not only optimize the disparity labels but also enforce piecewise smoothness of a normal map. Our deep learning approach is based on the assumption that if mathematical models allow us to compute a disparity and intrinsic components from a light field, then these models can be approximated with a deep convolutional neural network. Moreover, because disparity estimation and intrinsic light fields are closely related, a single network can be sufficient to perform all tasks together and they can benefit from each other. Thus, we establish a multi-task learning strategy for light fields, which is not only limited to the particular collection of tasks but (in theory) can also be used for various computer vision applications. We demonstrate the advantage of our approach on four state-of-the-art computer vision problems: disparity estimation, reflection separation, intrinsic images, and super-resolution. Extensive evaluations based on multiple, publicly-available, synthetic and real-world datasets prove our methodology and show the advantage of using light fields over other data structures. Our proposed algorithms outperform state-of-the-art methods for intrinsic images and disparity estimation, and achieve a competing quality for super-resolution and reflection separation.
Light Field Intrinsics With a Deep Encoder-Decoder Network
We present a fully convolutional autoencoder for light fields, which jointly encodes stacks of horizontal and vertical epipolar plane images through a deep network of residual layers. The complex structure of the light field is thus reduced to a comparatively low-dimensional representation, which can be decoded in a variety of ways. The different pathways of upconvolution we currently support are for disparity estimation and separation of the lightfield into diffuse and specular intrinsic components. The key idea is that we can jointly perform unsupervised training for the autoencoder path of the network, and supervised training for the other decoders. This way, we find features which are both tailored to the respective tasks and generalize well to datasets for which only example light fields are available. We provide an extensive evaluation on synthetic light field data, and show that the network yields good results on previously unseen real world data captured by a Lytro Illum camera and various gantries.
A Variational Model for Intrinsic Light Field Decomposition
We present a novel variational model for intrinsic light field decomposition, which is performed on four-dimensional ray space instead of a traditional 2D image. As most existing intrinsic image algorithms are designed for Lambertian objects, their performance suffers when considering scenes which exhibit glossy surfaces. In contrast, the rich structure of the light field with many densely sampled views allows us to cope with non-Lambertian objects by introducing an additional decomposition term that models specularity. Regularization along the epipolar plane images further encourages albedo and shading consistency across views. In evaluations of our method on real-world data sets captured with a Lytro Illum plenoptic camera, we demonstrate the advantages of our approach with respect to intrinsic image decomposition and specular removal.