Publikation:

Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields

Lade...
Vorschaubild

Dateien

Johannsen_2-apu5hpvjie2t3.pdf
Johannsen_2-apu5hpvjie2t3.pdfGröße: 52.15 MBDownloads: 37

Datum

2022

Autor:innen

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

DOI (zitierfähiger Link)
ArXiv-ID

Internationale Patentnummer

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Open Access Green
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Dissertation
Publikationsstatus
Published

Erschienen in

Zusammenfassung

One of the primary tasks of computer vision is to estimate the 3D structure of a scene from a set of given images. For stereo-based systems, this is fundamentally performed by identifying corresponding points within the respective views and triangulating their positions in 3D space. The most challenging part of this pipeline is identifying the correspondences. This works with little difficulty for the well-textured and Lambertian parts of the scene. Lambertian materials are defined as materials whose appearance is independent of the viewing angle, i.e., they look the same from different perspectives. The task gets more complicated if these two criteria are not met. Textureless regions are usually handled with the help of regularization, i.e., by imposing assumptions on the 3D structure of the scene. In the case of specular - i.e., non-Lambertian - materials, additional constraints on the reflective behavior from different angles are employed. This gets increasingly more challenging for materials that exhibit a high amount of gloss, i.e., where the surroundings are visibly reflected on the surface. Here, the matching may actually match points from the surrounding scene instead of the observed surface. In this thesis, I will discuss the ways in which depth estimation is still possible in the presence of such strong reflections. This setting is equivalent to observing an object through a semitransparent surface, such as dirty or stained glass. For the analysis of such scenes, I will utilize light fields. These correspond to a dense sampling of rays emerging from the scene, from slightly different perspectives. The structure of the light field facilitates the reformulation of the task of depth estimation as a problem of orientation analysis by representing the light field as epipolar plane images (EPIs). Given a scene where, for instance, an object is reflected on a planar surface of polished marble, the EPIs will show two superimposed line patterns in these regions. The orientations of these lines correspond to the depths of the reflecting surface and the reflected object. Similar observations are made in the case of semitransparent materials. After a formal definition of light fields and light-surface interactions, I discuss ways to estimate the orientation of the line patterns present in EPIs, using sparse coding and a depth-aware dictionary. This method is capable of reliably detecting regions with superimposed layers and estimating the disparity of the respective layers. In the second step, I will show how the luminosity of the individual layers can be reconstructed given this orientation information. Both of these were previously impossible with traditional camera techniques. The main goal of this thesis is to perform a 3D reconstruction of scenes with multilayered parts. For this task, light fields from different perspectives are combined to formulate a variational segmentation problem representing the scene structure. However, before this problem can be formulated, the relative positions of the individual light fields need to be known. I will show how a linear structure-from-motion algorithm tailored to light fields is formulated. This can be refined in a non-linear bundle adjustment step to jointly estimate the positions of all light fields capturing the scene. Given these positions and the individual multi-layered depth maps, a 3D cost volume is built, which can be optimized in a globally optimal way to yield a reconstruction of the scene. Here, specific focus is laid upon the reconstruction of very thin objects, similar to what would occur if a scene contains semitransparent glass.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
004 Informatik

Schlagwörter

Konferenz

Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690JOHANNSEN, Ole, 2022. Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields [Dissertation]. Konstanz: University of Konstanz
BibTex
@phdthesis{Johannsen2022Varia-69098,
  year={2022},
  title={Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields},
  author={Johannsen, Ole},
  address={Konstanz},
  school={Universität Konstanz}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/69098">
    <dcterms:abstract>One of the primary tasks of computer vision is to estimate the 3D structure of a scene from a
set of given images. For stereo-based systems, this is fundamentally performed by identifying
corresponding points within the respective views and triangulating their positions in 3D space. The
most challenging part of this pipeline is identifying the correspondences. This works with little
difficulty for the well-textured and Lambertian parts of the scene. Lambertian materials are defined
as materials whose appearance is independent of the viewing angle, i.e., they look the same from
different perspectives. The task gets more complicated if these two criteria are not met. Textureless
regions are usually handled with the help of regularization, i.e., by imposing assumptions on the
3D structure of the scene. In the case of specular - i.e., non-Lambertian - materials, additional
constraints on the reflective behavior from different angles are employed. This gets increasingly
more challenging for materials that exhibit a high amount of gloss, i.e., where the surroundings are
visibly reflected on the surface. Here, the matching may actually match points from the surrounding
scene instead of the observed surface. In this thesis, I will discuss the ways in which depth estimation
is still possible in the presence of such strong reflections. This setting is equivalent to observing an
object through a semitransparent surface, such as dirty or stained glass.
For the analysis of such scenes, I will utilize light fields. These correspond to a dense sampling of
rays emerging from the scene, from slightly different perspectives. The structure of the light field
facilitates the reformulation of the task of depth estimation as a problem of orientation analysis
by representing the light field as epipolar plane images (EPIs). Given a scene where, for instance, an
object is reflected on a planar surface of polished marble, the EPIs will show two superimposed line
patterns in these regions. The orientations of these lines correspond to the depths of the reflecting
surface and the reflected object. Similar observations are made in the case of semitransparent
materials.
After a formal definition of light fields and light-surface interactions, I discuss ways to estimate the
orientation of the line patterns present in EPIs, using sparse coding and a depth-aware dictionary.
This method is capable of reliably detecting regions with superimposed layers and estimating
the disparity of the respective layers. In the second step, I will show how the luminosity of the
individual layers can be reconstructed given this orientation information. Both of these were
previously impossible with traditional camera techniques.
The main goal of this thesis is to perform a 3D reconstruction of scenes with multilayered parts.
For this task, light fields from different perspectives are combined to formulate a variational
segmentation problem representing the scene structure. However, before this problem can be
formulated, the relative positions of the individual light fields need to be known. I will show how a
linear structure-from-motion algorithm tailored to light fields is formulated. This can be refined in
a non-linear bundle adjustment step to jointly estimate the positions of all light fields capturing
the scene. Given these positions and the individual multi-layered depth maps, a 3D cost volume is
built, which can be optimized in a globally optimal way to yield a reconstruction of the scene. Here,
specific focus is laid upon the reconstruction of very thin objects, similar to what would occur if a
scene contains semitransparent glass.</dcterms:abstract>
    <dc:language>eng</dc:language>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/69098/4/Johannsen_2-apu5hpvjie2t3.pdf"/>
    <dc:contributor>Johannsen, Ole</dc:contributor>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/69098/4/Johannsen_2-apu5hpvjie2t3.pdf"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:rights>terms-of-use</dc:rights>
    <dcterms:title>Variational 3D Reconstruction of Non-Lambertian Scenes Using Light Fields</dcterms:title>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2024-01-18T11:33:30Z</dc:date>
    <dc:creator>Johannsen, Ole</dc:creator>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2024-01-18T11:33:30Z</dcterms:available>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:issued>2022</dcterms:issued>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/69098"/>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

February 15, 2022
Hochschulschriftenvermerk
Konstanz, Univ., Diss., 2022
Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Diese Publikation teilen