Publikation: Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields
Dateien
Datum
Autor:innen
Herausgeber:innen
ISSN der Zeitschrift
Electronic ISSN
ISBN
Bibliografische Daten
Verlag
Schriftenreihe
Auflagebezeichnung
URI (zitierfähiger Link)
Internationale Patentnummer
Link zur Lizenz
Angaben zur Forschungsförderung
Projekt
Open Access-Veröffentlichung
Core Facility der Universität Konstanz
Titel in einer weiteren Sprache
Publikationstyp
Publikationsstatus
Erschienen in
Zusammenfassung
One of the primary tasks of computer vision is to estimate the 3D structure of a scene from a set of given images. For stereo-based systems, this is fundamentally performed by identifying corresponding points within the respective views and triangulating their positions in 3D space. The most challenging part of this pipeline is identifying the correspondences. This works with little difficulty for the well-textured and Lambertian parts of the scene. Lambertian materials are defined as materials whose appearance is independent of the viewing angle, i.e., they look the same from different perspectives. The task gets more complicated if these two criteria are not met. Textureless regions are usually handled with the help of regularization, i.e., by imposing assumptions on the 3D structure of the scene. In the case of specular - i.e., non-Lambertian - materials, additional constraints on the reflective behavior from different angles are employed. This gets increasingly more challenging for materials that exhibit a high amount of gloss, i.e., where the surroundings are visibly reflected on the surface. Here, the matching may actually match points from the surrounding scene instead of the observed surface. In this thesis, I will discuss the ways in which depth estimation is still possible in the presence of such strong reflections. This setting is equivalent to observing an object through a semitransparent surface, such as dirty or stained glass. For the analysis of such scenes, I will utilize light fields. These correspond to a dense sampling of rays emerging from the scene, from slightly different perspectives. The structure of the light field facilitates the reformulation of the task of depth estimation as a problem of orientation analysis by representing the light field as epipolar plane images (EPIs). Given a scene where, for instance, an object is reflected on a planar surface of polished marble, the EPIs will show two superimposed line patterns in these regions. The orientations of these lines correspond to the depths of the reflecting surface and the reflected object. Similar observations are made in the case of semitransparent materials. After a formal definition of light fields and light-surface interactions, I discuss ways to estimate the orientation of the line patterns present in EPIs, using sparse coding and a depth-aware dictionary. This method is capable of reliably detecting regions with superimposed layers and estimating the disparity of the respective layers. In the second step, I will show how the luminosity of the individual layers can be reconstructed given this orientation information. Both of these were previously impossible with traditional camera techniques. The main goal of this thesis is to perform a 3D reconstruction of scenes with multilayered parts. For this task, light fields from different perspectives are combined to formulate a variational segmentation problem representing the scene structure. However, before this problem can be formulated, the relative positions of the individual light fields need to be known. I will show how a linear structure-from-motion algorithm tailored to light fields is formulated. This can be refined in a non-linear bundle adjustment step to jointly estimate the positions of all light fields capturing the scene. Given these positions and the individual multi-layered depth maps, a 3D cost volume is built, which can be optimized in a globally optimal way to yield a reconstruction of the scene. Here, specific focus is laid upon the reconstruction of very thin objects, similar to what would occur if a scene contains semitransparent glass.
Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
Schlagwörter
Konferenz
Rezension
Zitieren
ISO 690
JOHANNSEN, Ole, 2022. Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields [Dissertation]. Konstanz: University of KonstanzBibTex
@phdthesis{Johannsen2022Varia-69098, year={2022}, title={Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields}, author={Johannsen, Ole}, address={Konstanz}, school={Universität Konstanz} }
RDF
<rdf:RDF xmlns:dcterms="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:bibo="http://purl.org/ontology/bibo/" xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:void="http://rdfs.org/ns/void#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/69098"> <dcterms:abstract>One of the primary tasks of computer vision is to estimate the 3D structure of a scene from a set of given images. For stereo-based systems, this is fundamentally performed by identifying corresponding points within the respective views and triangulating their positions in 3D space. The most challenging part of this pipeline is identifying the correspondences. This works with little difficulty for the well-textured and Lambertian parts of the scene. Lambertian materials are defined as materials whose appearance is independent of the viewing angle, i.e., they look the same from different perspectives. The task gets more complicated if these two criteria are not met. Textureless regions are usually handled with the help of regularization, i.e., by imposing assumptions on the 3D structure of the scene. In the case of specular - i.e., non-Lambertian - materials, additional constraints on the reflective behavior from different angles are employed. This gets increasingly more challenging for materials that exhibit a high amount of gloss, i.e., where the surroundings are visibly reflected on the surface. Here, the matching may actually match points from the surrounding scene instead of the observed surface. In this thesis, I will discuss the ways in which depth estimation is still possible in the presence of such strong reflections. This setting is equivalent to observing an object through a semitransparent surface, such as dirty or stained glass. For the analysis of such scenes, I will utilize light fields. These correspond to a dense sampling of rays emerging from the scene, from slightly different perspectives. The structure of the light field facilitates the reformulation of the task of depth estimation as a problem of orientation analysis by representing the light field as epipolar plane images (EPIs). Given a scene where, for instance, an object is reflected on a planar surface of polished marble, the EPIs will show two superimposed line patterns in these regions. The orientations of these lines correspond to the depths of the reflecting surface and the reflected object. Similar observations are made in the case of semitransparent materials. After a formal definition of light fields and light-surface interactions, I discuss ways to estimate the orientation of the line patterns present in EPIs, using sparse coding and a depth-aware dictionary. This method is capable of reliably detecting regions with superimposed layers and estimating the disparity of the respective layers. In the second step, I will show how the luminosity of the individual layers can be reconstructed given this orientation information. Both of these were previously impossible with traditional camera techniques. The main goal of this thesis is to perform a 3D reconstruction of scenes with multilayered parts. For this task, light fields from different perspectives are combined to formulate a variational segmentation problem representing the scene structure. However, before this problem can be formulated, the relative positions of the individual light fields need to be known. I will show how a linear structure-from-motion algorithm tailored to light fields is formulated. This can be refined in a non-linear bundle adjustment step to jointly estimate the positions of all light fields capturing the scene. Given these positions and the individual multi-layered depth maps, a 3D cost volume is built, which can be optimized in a globally optimal way to yield a reconstruction of the scene. Here, specific focus is laid upon the reconstruction of very thin objects, similar to what would occur if a scene contains semitransparent glass.</dcterms:abstract> <dc:language>eng</dc:language> <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/> <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/69098/4/Johannsen_2-apu5hpvjie2t3.pdf"/> <dc:contributor>Johannsen, Ole</dc:contributor> <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/69098/4/Johannsen_2-apu5hpvjie2t3.pdf"/> <foaf:homepage rdf:resource="http://localhost:8080/"/> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dc:rights>terms-of-use</dc:rights> <dcterms:title>Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields</dcterms:title> <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2024-01-18T11:33:30Z</dc:date> <dc:creator>Johannsen, Ole</dc:creator> <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2024-01-18T11:33:30Z</dcterms:available> <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dcterms:issued>2022</dcterms:issued> <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/69098"/> </rdf:Description> </rdf:RDF>